Project

General

Profile

Converting NetCDF to GRIB

Added by Russell Glazer over 9 years ago

Hello,

I have a file that I can successfully convert from .nc to .grb using the format 'cdo -f grb file.nc file.grb'.

https://www.dropbox.com/s/ynaqe9jubhi2t7p/add_hifreq_3_8.nc
https://www.dropbox.com/s/ljnr4tiklknj6we/add_hifreq_3_8.grb

But a problem for some reason arises when I try to convert a larger file. This is pretty much the exact same file just bigger. The cdo operation finishes without any error messages but the grb file that is produced is not usable and wasn't converted correctly.

https://www.dropbox.com/s/nv0eg2w7h6cu0ws/sstatlantic_addhifreq.nc
https://www.dropbox.com/s/8bii9laa4gn07at/sstatlantic_addhifreq.grb

I am using cdo in a unix environment so I don't see how memory space would be an issue. Any help is greatly appreciated thank you

-Russell


Replies (6)

RE: Converting NetCDF to GRIB - Added by Jaison-Thomas Ambadan over 9 years ago

I do not see any problem(?). Please post the output of "cdo -V". See the attached figure from the first timestep of the output grib

cdo -v -f grb -copy sstatlantic_addhifreq.nc ofile.grb
cdo grfill,min=290.0,max=320,count=10,RGB="TRUE",device="png" -seltimestep,1 ofile.grb sstatlantic_addhifreq

RE: Converting NetCDF to GRIB - Added by Russell Glazer over 9 years ago

$ cdo -V
Climate Data Operators version 1.6.1 (http://code.zmaw.de/projects/cdo)
Compiler: gcc -std=gnu99 -O2 -m64 -mtune=generic -fPIC -pthread
version: gcc (GCC) 4.4.6 20120305 (Red Hat 4.4.6-4)
Compiled: by paulvdm on build (x86_64-unknown-linux-gnu) Sep 5 2013 16:39:30
with: PTHREADS NC4 OPeNDAP Z UDUNITS2 PROJ.4 XML2
filetype: srv ext ieg grb grb2 nc nc2 nc4 nc4c
CDI library version : 1.6.1 of Sep 5 2013 16:39:24
CGRIBEX library version : 1.6.1 of Jun 27 2013 15:38:33
GRIB_API library version : 1.11.0
netCDF library version : 4.1.3 of Nov 20 2012 11:37:38 $
HDF5 library version : 1.8.7
SERVICE library version : 1.3.1 of Sep 5 2013 16:39:22
EXTRA library version : 1.3.1 of Sep 5 2013 16:39:22
IEG library version : 1.3.1 of Sep 5 2013 16:39:22
FILE library version : 1.8.2 of Sep 5 2013 16:39:22

RE: Converting NetCDF to GRIB - Added by Russell Glazer over 9 years ago

I downloaded your file ofile.grb and when I use a quick program to check the GRIB format of the file it shows it to not be a completed GRIB file. This quick program is a utility that checks the contents of a GRIB file and shows the code and ltype of the variables in the file. Here are the results when I use it on ofile.grb:

[rhg11c@coaps ungrib]$ ./g1print.exe ../../../inputdata/OSTIA/ofile.grb
Copen: File = ../../../inputdata/OSTIA/ofile.grb z2(A
Fortran Unit = 4810112
UNIX File descriptor: 3

----------------------------------------------------
rec GRIB GRIB Lvl Lvl Lvl Time Fcst
Num Code name Code one two hour
----------------------------------------------------
End-of-record mark (7777) not found-479881758
Sec0(1) = 8479809 0
1 1 1 0 0 2010-09-08_00:00 + 00
  • stopping in findgrib in gribcode ***
I could not find the GRIB string in the input file
after testing the first 100,000 bytes.
The file may be corrupt or it is not a GRIB file.
Perhaps a gzipped GRIB file or netcdf?

Warning: ieee_inexact is signaling
findgrib

If you tried displaying or opening this file I don't think it will work. But like I said before, I did not have this problem when I converted add_hifreq_3_8.nc to grb.

RE: Converting NetCDF to GRIB - Added by Jaison-Thomas Ambadan over 9 years ago

Have a look at (Uwe's reply): https://code.zmaw.de/boards/1/topics/1144?r=1173#message-1173

your input NetCDF file doesn't provide all the required meta-data for the GRIB (such as level type, table number etc, although they are not required for CDO to operate).

You could try -setmissval,0 ie.

cdo -f grb -setmissval,0 sstatlantic_addhifreq.nc ofile.grb

now try again with your program (I can also open/see with Panoply)

RE: Converting NetCDF to GRIB - Added by Uwe Schulzweida over 9 years ago

The large GRIB file has data with 2602x1402=3648004 grid points. CDO stores netCDF double precision floats to 24bit packed GRIB per default. The result is a GRIB record with 10944120 bytes. The max. record length in the GRIB1 standard is 8388607 bytes. CDO uses the ECMWF extension of the GRIB1 standard for larger GRIB records. These large GRIB records can only be read by CDO or tools from the ECMWF.
You can create a 16bit GRIB record to reduce the record size:

cdo -b 16 -f grb copy file.nc file.grb
This will produce a standard GRIB1 file.
Jaison's suggestion could be also useful with your data:
cdo -b 16 -f grb setmissval,0 file.nc file.grb

RE: Converting NetCDF to GRIB - Added by Russell Glazer over 9 years ago

Ok, this seems to have solved the problem. Thank you very much! I was stuck on this for several weeks

    (1-6/6)