Importing lat lon value data from text before interpolating
Added by Peter Jacobs over 4 years ago
Hello! First of all, thanks to all who work on CDO. It's a wonderful tool that I use regularly and am citing in my forthcoming works.
I have point values for sea surface temperatures, along with their lat and lon coordinates, in a CSV file (LGM_coords_SSTs.csv).
I assume there is no way to read this directly into CDO based on previous readings, but I am unsure about this.
I am hoping to get these into CDO, mask out the land areas using an existing netcdf file, and then interpolate these point estimates.
In theory, I know how to do the second and third of these steps already (masking and interpolating).
I have been working on this for ages and can't figure it out.
For ingesting the lat lon data, based on previous answers from this forum I have tried:
cdo -f nc input,my_gridfile ofile.nc < my_ascii_data
I am completely agnostic as to how to accomplish this in terms of importing the data. Any help would be incredibly appreciated as I am working on a crazy deadline.
Thank you!
my_gridfile (1.32 KB) my_gridfile | lat and lon values | ||
my_ascii_data (1.35 KB) my_ascii_data | SST values | ||
LGM_coords_SSTs.csv (4.04 KB) LGM_coords_SSTs.csv |
Replies (7)
RE: Importing lat lon value data from text before interpolating - Added by Uwe Schulzweida over 4 years ago
Your data file my_ascii_data contains 140 points and the grid description file contains 140 lon/lat coordinates for this point.
You have to replace the following lines in my_gridfile:
xsize = 11 ysize = 11by
gridsize = 140
RE: Importing lat lon value data from text before interpolating - Added by Peter Jacobs over 4 years ago
Thanks very much! This did allow me to create the nc file, but attempting to read the values resulted in the attached error from Panoply about non-unique lon values.
I have datapoints along the same lat and same lon points (but not both).
It also looks like I will be unable to regrid properly because the grid is unstructured?
So I assume I need to do this in two steps somehow- define a regular grid that would encompass all the data points, and only then pass the actual lat, lon, and variable values?
Thanks!
RE: Importing lat lon value data from text before interpolating - Added by Peter Jacobs over 4 years ago
I should add that I can manipulate these data in R prior to dealing with them in CDO.
So if it in any way helps, I can save the data as a raster first.
RE: Importing lat lon value data from text before interpolating - Added by Peter Jacobs over 4 years ago
Sorry for the multiple posts-
I tried writing the points out as a netcdf from R, with the attached result (r001.nc). Checking the file with ncdump and opening in Panoply seem to confirm that this worked.
However, prior to writing the file I tried to set the missing values to -999 rather than NaNs to save the trouble with masking. My goal now is to mask out the land values and then interpolate over the SSTs only.
I assume then I need to do a two step process for dealing with the mask and missing values.
Per the tutorial: https://code.mpimet.mpg.de/projects/cdo/wiki/Tutorial#Missing-values
I should
1. set the current missing values to -9999 or similar
2. then mask out land values to NaNs using an existing netcdf
3. then set the -9999 back to missing
4. interpolate the SST values
and this should result in interpolated SSTs values in the ocean only?
RE: Importing lat lon value data from text before interpolating - Added by Peter Jacobs over 4 years ago
And I have the attached mask file if that helps, although here I am stuck because the grid sizes are apparently incompatible even though I originally created the r001.nc using the raster data I used to create the mask....
Seems like the grids are different, but if I attempt to use the point based nc file for the grid to regrid my mask, it just NaNs everything.
For clarity I am renaming the files:
- LGM-SSTs_from-raster.nc: a netcdf of point estimate SSTs exported from R
- mask_360x180.nc: the land sea mask I was hoping to be able to apply to the above before interpolating the point values (i.e. force land-only to NaNs/missing)
RE: Importing lat lon value data from text before interpolating - Added by Uwe Schulzweida over 4 years ago
Yes, the grids are different. You can either use remapnn on the unstructured data
cdo -f nc input,my_gridfile LGM_SST.nc < my_ascii_data REMAP_EXTRAPOLATE=off CDO_GRIDSEARCH_RADIUS=0.5 cdo remapnn,mask_360x180.nc LGM_SST.nc LGM_SST_360x180.ncor on the raster data from R to remap the data to 360x180
REMAP_EXTRAPOLATE=off cdo remapnn,mask_360x180.nc LGM-SSTs_from-raster.nc LGM_SST_360x180.ncThen you can use setmisstonn or setmisstodis to fill the missing areas. Apply the masking of the land at the end:
cdo setmisstodis LGM_SST_360x180.nc tmpfile cdo mul tmpfile mask_360x180.nc result rm tmpfileOr in one line:
cdo -mul -setmisstodis LGM_SST_360x180.nc mask_360x180.nc result
RE: Importing lat lon value data from text before interpolating - Added by Peter Jacobs over 4 years ago
Amazing, thank you so much!
I am incredibly grateful!