Spatial merge of overlapping datasets, to update values in old data
Added by P Ruess over 3 years ago
Hello,
I am trying to spatially merge two datasets: forest_conus_2008-2010.nc and forest_global.nc (see attached).
Desired outcome: A global dataset (or at least CONUS, Canada, and Mexico) which contains data from forest_conus_2008-2010.nc in all grid-cells where that data is available, while the rest of the world should always contain the time-invariant data from forest_global.nc. So for 2008, 2009, and 2010, all values outside of the CONUS should be time-invariant, while CONUS values will slightly change annually.
What I've tried: I have tried replacing the global dataset with the CONUS data where those values are available, using "cdo replace". This threw grid size errors, which I tried to address by extending the grid of the CONUS dataset to match the global dataset (filled with 1.e20). I guess because this fill had values of 1.e20, rather than NA, I ended up with the entire global dataset being replaced with these filler values globally, so nothing was improved. I also tried using "cdo mergegrid" but couldn't quite figure how to make it work, since my global dataset overlaps with the CONUS dataset.
Some posts I used for ideas, to no avail:
- mergegrid: https://code.mpimet.mpg.de/boards/1/topics/26
- replace: https://code.mpimet.mpg.de/boards/2/topics/6567
Final thoughts:
- All my variable names match, so this shouldn't be a problem.
- "cdo mask" seems like it may provide an option, but I'm confused about how to use it for removal rather than selection (ie. I want to remove the CONUS region from the global dataset, rather than selecting a specific region). This post seems useful but I don't quite understand the methodology: https://code.mpimet.mpg.de/boards/2/topics/5036
Any help is greatly appreciated, thank you!
| forest_conus_2008-2010.nc (1.09 MB) forest_conus_2008-2010.nc | CONUS-specific updated annual forest data | ||
| forest_global.nc (35.6 MB) forest_global.nc | Global older time-invariant forest data |
Replies (3)
RE: Spatial merge of overlapping datasets, to update values in old data - Added by Uwe Schulzweida over 3 years ago
I think mergegrid does basically what you want.
The first inputfile must be the global one. Since this has only one time step and the other file contains 3 years the operator duplicate,3 must be added. Then you have to set the time again, because it is not present in the first file:
cdo settaxis,2008-01-01,0,1year -mergegrid -duplicate,3 forest_global.nc forest_conus_2008-2010.nc result
RE: Spatial merge of overlapping datasets, to update values in old data - Added by P Ruess over 3 years ago
Thank you, this worked!
These is now a new problem though: the year 2010 has zeros outside of the CONUS domain in some places, and these are now replacing the global data for 2010.
Is there a way to mask the 2010 data to remove data points not present in the 2008 or 2009 data (these two years should be identical)?
I followed advice from here but was unable to remove these outside points: https://code.mpimet.mpg.de/boards/2/topics/5036
I can post a new question if that's preferable, seeing as this isn't topically relevant anymore.
cdo seltimestep,1 forest_conus_2008-2010.nc forest_2008.nc
cdo mul -selname,fracVegCover forest_2008-2010.nc forest_2008.nc forest_2008-2010_masked.nc
cdo remapnn,forest_2008.nc -selname,fracVegCover forest_2008-2010.nc forest_2008-2010_alt.nc cdo mul forest_2008-2010_alt.nc forest_2008.nc forest_2008-2010_alt_masked.nc
I have attached output files from both attempts in case that is useful.
RE: Spatial merge of overlapping datasets, to update values in old data - Added by P Ruess over 3 years ago
Nevermind, I figured this out using a mask as follows:
Select only 2008 as base year for masking
cdo seltimestep,1 forest_conus_2008-2010.nc forest_2008.nc
Define masking file by changing all values [0,1] to 1
cdo setrtoc,0,1,1 forest_2008.nc forest_2008_mask.nc
Use mask to remove the noise outside of the CONUS region
cdo -ifthen -gtc,0 forest_2008_mask.nc forest_2008-2010.nc forest_2008-2010_edit.nc