Project

General

Profile

Not able to fix error: "Reference to source grid not found!"

Added by N. A. almost 2 years ago

In order to transform DWD data from ICON grid format into lon-lat grid, I downloaded & set up cdo with all needed additions (following these instructions: https://www.studytrails.com/2012/12/28/install-climate-data-operator-cdo-with-netcdf-grib2-and-hdf5-support/, at some points I had to adjust some steps since the tutorial is already more than 10 years old, but basically it did help me through the process well). Now I should be ready to do the transformation. Though I'm stuck at the following point: When I want to calculate the interpolation weights as needed to be done in advance, I run the following command:

cdo gennn,target_grid_icon_d2_002.txt icon_grid_0047_R19B07_L.nc weights_icon_d2_002.nc

I downloaded the 3 files included in this command here: https://opendata.dwd.de/weather/lib/cdo/. And as far as I can see, they should match together (meaning that the folder "ICON_D2_002_EASY.tar.bz2" including the .txt and weights file should match with the respective icon_grid file you can see in the command). Though the output is just:

Reference to source grid not found!

I already did google the issue and also did find a promising thread right here: https://code.mpimet.mpg.de/boards/1/topics/9855. Though I was not really able to apply the suggested solutions there to my problem successfully. Also the "-selgrid,2" option did not really help in my case. This is why I decided to open my own thread about it.

(In order to trying to understand by myself what the error message could mean I looked at the grid file information by ncdump -h. Comparing it with another random grid file from the same opendata source (icon_grid_0026_R03B07_L.nc in my case) I realized a significant difference - while in the 0047_R19B07 file I want to work with it reads:

:source = "Unknown, branch Unknown, rev. Unknown"

in the 0026_R03B07 file I found:

:source = "icon-dev\b" ;
:uuidOfHGrid = "a27b8de6-18c4-11e4-820a-b5b098c6a5c0"

with the "uuidOfHGrid" line not even included in the header of the 0047_R19B07 file. I thought that this could maybe be related to the issue since the error message is about a missing source (if I got it right). I thought about adding the source information to the 0047_R19B07 file (if possible) but I don't think this will help solve the issue because I have no idea about what "uuidOfHGrid" should be in this case.)

I hope this information is sufficient to get an idea about my issue and maybe to find a solution. (Since it's my first time posting here I'm not entirely sure yet if all relevant information are included. If not, just feel free to ask. :))


Replies (6)

RE: Not able to fix error: "Reference to source grid not found!" - Added by Uwe Schulzweida almost 2 years ago

Hello Niklas,

With selgrid,2 it works for me without any problems and that is also the only solution, because the other grids in the input file have no coordinates:

cdo gennn,target_grid_icon_d2_002.txt -selgrid,2 ../icon_grid_0047_R19B07_L.nc weights_icon_d2_002.nc

Cheers,
Uwe

RE: Not able to fix error: "Reference to source grid not found!" - Added by N. A. almost 2 years ago

Hello Uwe, thanks for answering! The bug is still there, but your reply helped me to localize it more precisely. I apologize for being wrong at the respective point.
What I forgot to mention is that in the first attempt I didn't execute the command at its own but included in a bash script (inspired from https://www.dwd.de/DE/leistungen/opendata/help/modelle/Opendata_cdo_DE.pdf?__blob=publicationFile&v=4, page 3/4). I tried out your command on its own and indeed it does work. And I realized it also did in the bash script (it also does now after inserting "-selgrid,2"). So I was wrong thinking that it would fail at this step (line 10 in the script in the linked pdf).
The error message itself is referred to the actual remapping process which is line 16 in the script of the pdf. If you or someone else wants to check what I did you can take a grib2 file from opendata.dwd.de as example, in my case I tried with files from https://opendata.dwd.de/weather/nwp/icon-d2-eps/grib/00/u_10m/ and https://opendata.dwd.de/weather/nwp/icon-d2/grib/00/u_10m/ (to see if there is a difference between icon-d2-eps_germany[...] and icon-d2_germany[...] files, but in my case there isn't, the bug is the same in both cases). So, assuming all respective files are in the same folder (for simplicity), including the output file (test.grib2 in this case), the command would be e.g.:

cdo -f grb2 remap,target_grid_icon_d2_002.txt,weights_icon_d2_002.nc icon-d2-eps_germany_icosahedral_single-level_2023021700_000_2d_u_10m.grib2 test.grib2

And this just gives me the error message (as mentioned in my first post, but shown more in detail here):

cdo remap (Warning): Reference to horizontal grid not available!

cdo remap (Abort): Reference to source grid not found!

And this is the actual point where I'm stuck. Using cdo sinfov icon-d2-eps_germany_icosahedral_single-level_2023021700_000_2d_u_10m.grib2 I get (as part of larger file information):

[...]

Grid coordinates :
1 : unstructured : points=542040

[...]

At least this shows me that the grid size matches in all respective files and should not be the reason for the error. All I can say to my understanding is that missing "References" are the main point of the bug. But at the moment I have no further idea how to fix it... So I hope based onto this more detailed information you and/or someone else will be able to help at this point.

Thank you so far and best regards!

RE: Not able to fix error: "Reference to source grid not found!" - Added by Uwe Schulzweida almost 2 years ago

I tried it with the following data and it works fine on different machines from CDO version 2.0.0:

cdo -f grb2 remap,target_grid_icon_d2_002.txt,weights_icon_d2_002.nc icon-d2-eps_germany_icosahedral_single-level_2023022100_000_2d_u_10m.grib2 test.grib2
Which CDO version are you using?

RE: Not able to fix error: "Reference to source grid not found!" - Added by N. A. almost 2 years ago

Hi Uwe! Good question, indeed I could/should have given this information in advance. During my last tries I had been working with version 2.1.1, the most current one I was able to find then. Based onto what you wrote I replaced this version with the 2.0.0 one and gave it another try, but unfortunately this doesn't fix the error either...

I don't know if it's relevant but here I list you the installed versions of the other mandatory packages in the "background":

eccodes-2.28.0
hdf5-1.12.2
netcdf-4.9.1
zlib-1.2.13

...and some additional ones (optional and/or not included in the outdated tutorial I linked above):

curl-7.88.1
jasper-4.0.0
libaec-1.0.6 (Adaptive Entropy Coding)

Do you think it could be useful to replace one or several packages with other versions? Maybe my way of thinking "the more up to date the better" is not necessarily always true... ;-)

RE: Not able to fix error: "Reference to source grid not found!" - Added by Uwe Schulzweida almost 2 years ago

I suspect replacing libraries will not help.
Maybe the cdo verbose output will help (cdo -v remap...). For me, the relevant part looks like this:

cdo -f grb2 -v remap,target_grid_icon_d2_002.txt,weights_icon_d2_002.nc icon-d2-eps_germany_icosahedral_single-level_2023022100_000_2d_u_10m.grib2 test.grib2

cdo    remap: Set max_remaps to 2
cdo    remap: Reading remap weights: SCRIP remapping with CDO
cdo    remap: From file: weights_icon_d2_002.nc
cdo    remap: mapType = Nearest neighbor
cdo    remap: normalize_opt = none
cdo    remap: Remapping between: unstructured and lonlat
cdo    remap: Using remap 0

RE: Not able to fix error: "Reference to source grid not found!" - Added by N. A. almost 2 years ago

Hello Uwe, sorry for replying late!

I tried your "verbose" suggestion which did work as far as I remember, though it didn't really contribute to fix the issue. But it's solved meanwhile otherwise: When changing back from cdo version 2.0.0 back to 2.1.1 I had to download the package again - nothing special, but for whatever reason after this download and setting up the 2.1.1 version again as usual it's working now. My only explanation for this is that something must have went wrong during the former downloads which caused cdo to fail... I have no idea what but as long as it works, I will not speculate about further details and hope it stays this way now. ;)
Thank you for your support though, I learned some further backgrounds about all this and maybe in case of other bugs I can come back to it and maybe find solutions on my own before asking again here. ;-)

Best regards!

    (1-6/6)