Project

General

Profile

Forums » Support »

Remapbil with lat/lon binary files

Added by Sentia Goursaud almost 4 years ago

Hello everyone,

I wish I could convert the grid of my netcdf files to equal area grid. The only solution I would was to make a bilinear interpolation using the EASE-Grid data (https://nsidc.org/data/ease/tools#geo_data_files). However, the grid consists in two binary files (EASE2_M36km.lons.964x406x1.double and EASE2_M36km.lats.964x406x1.double). I tried to make:

cdo remapbil,lon=EASE2_M36km.lons.964x406x1.double/lat=EASE2_M36km.lats.964x406x1.double ifile.nc ofile.nc

but It was returned that the lat lon files were not readable. Being not familiar with binary files, I tried to convert it to ascii files and make the same command with the ascii files instead, but still did not work.

Finally, I saw this post: https://code.mpimet.mpg.de/boards/2/topics/96
I could write such a target file, but I do not manage to read the needed information to write the ascii file.

Thanks in advance to the people that will answer me, and sorry for so beginner questions.


Replies (8)

RE: Remapbil with lat/lon binary files - Added by Karin Meier-Fleischer almost 4 years ago

Please, upload the lats and lons files.

RE: Remapbil with lat/lon binary files - Added by Karin Meier-Fleischer almost 4 years ago

Okay, that was also very interesting and instructive for me. Since CDO cannot do this, but NCL.

The NCL script to read the latitude and longitude values and write them to a netCDF file (with one timestep and dummy data):

NCL script script.ncl to convert the EASE lats and lons to netCDF:

flats = "EASE2_M36km.lats.964x406x1.double" 
flons = "EASE2_M36km.lons.964x406x1.double" 

nlat = 406
nlon = 964

setfileoption ("bin", "ReadByteOrder", "LittleEndian")

glat  = fbindirread (flats, 0, -1, "double")
glon  = fbindirread (flons, 0, -1, "double")

lon = glon(0:nlon-1)
lat = glat(0::nlon)

lon!0         = "lon" 
lon@long_name = "lon" 
lon@units     = "degrees_east" 
lon&lon       = lon

lat!0         = "lat" 
lat@long_name = "lat" 
lat@units     = "degrees_north" 
lat&lat       = lat
lat = lat(::-1)

;-- dummy time and data
time   = cd_inv_calendar(2020,6,30,12,0,0,"hours since 2020-01-01 00:00:00",0)
time!0 = "time" 
time@long_name = "time" 

data = new((/1,nlat,nlon/), "float")
data!0 = "time" 
data!1 = "lat" 
data!2 = "lon" 
data&time = time
data&lat  = lat
data&lon  = lon

;-- write grid and dummy data to netCDF file
setfileoption("nc","Format","NetCDF4Classic")
system("rm -rf EASE2_M36km.lats.964x406_grid.nc")
fout = addfile("EASE2_M36km.lats.964x406_grid.nc","c")

setfileoption(fout, "DefineMode", True)

fAtt               =  True        ;-- assign file attributes
fAtt@title         = "Convert binary to netCDF"  
fAtt@source_file   = "EASE2_M36km.lats.964x406x1.double; EASE2_M36km.lons.964x406x1.double" 
fAtt@creation_date =  systemfunc ("date")          
fileattdef(fout,fAtt) 

dimNames = (/"time", "lat", "lon"/)  
dimSizes = (/ -1   ,  nlat,  nlon /) 
dimUnlim = (/ True , False, False/)   
filedimdef(fout,dimNames,dimSizes,dimUnlim)

filevardef(fout, "time", typeof(time), getvardims(time)) 
filevardef(fout, "lat" , typeof(lat),  getvardims(lat))                          
filevardef(fout, "lon" , typeof(lon),  getvardims(lon))                          
filevardef(fout, "data", typeof(data), getvardims(data))

filevarattdef(fout, "time", time)
filevarattdef(fout, "lat" , lat)
filevarattdef(fout, "lon" , lon)
filevarattdef(fout, "data", data)

setfileoption(fout,"DefineMode",False)

fout->lat  = (/lat/)
fout->lon  = (/lon/)
fout->data = (/data/)
fout->time = (/time/)

Run the NCL script:

ncl script.ncl

Remap your data file to the EASE grid with CDO:

cdo -remapnn,EASE2_M36km.lats.964x406_grid.nc infile.nc outfile.nc

RE: Remapbil with lat/lon binary files - Added by Sentia Goursaud almost 4 years ago

Wow thank you very much !!! It worked perfectly.

I am not familiar with ncl at all, so very instructive for me as well.

Thank you again,
Sentia

RE: Remapbil with lat/lon binary files - Added by Sentia Goursaud almost 4 years ago

Hello,

Sorry for the one more message.

I am actually focused on Antarctica, and using the M36 grid, I get a big hole in central Antarctica. So I decided to apply your magic script to the S36 grid (see the binary lat/lon files enclosed). I adapted it to the new files. However, it seems there is a problem. For instance, using python, here are the messages I get opening the file:

/packages/python/miniconda3/lib/python3.6/site-packages/iris/fileformats/_pyke_rules/compiled_krb/fc_rules_cf_fc.py:1952: UserWarning: Failed to create 'lat' dimension coordinate: The points array must be strictly monotonic.
Gracefully creating 'lat' auxiliary coordinate instead.
error=e_msg))
/packages/python/miniconda3/lib/python3.6/site-packages/iris/fileformats/_pyke_rules/compiled_krb/fc_rules_cf_fc.py:1952: UserWarning: Failed to create 'lon' dimension coordinate: The points array must be strictly monotonic.

And when applying a remapnn to my initial not-regridded output, althouth cut on the Southern Hemisphere (-90 to 0 degS), data from Antarctica disappear.

Hope you will be able to help me again.
Sentia

Download all files
EASE2_S36km.lats.500x500x1.double (1.91 MB) EASE2_S36km.lats.500x500x1.double
EASE2_S36km.lons.500x500x1.double (1.91 MB) EASE2_S36km.lons.500x500x1.double
script_S36km.ncl (1.97 KB) script_S36km.ncl

RE: Remapbil with lat/lon binary files - Added by Karin Meier-Fleischer almost 4 years ago

What about your input data, does it contain values over the Antarctica area? Or are the values missing values? Without your input data it is hard to say where the hole is coming from.

Sorry, but the python problem seems to come from iris package and I'm not an expert of that.

RE: Remapbil with lat/lon binary files - Added by Karin Meier-Fleischer almost 4 years ago

The binary EASE grid files contains missing values which is not correct.

od -Ad -e -N80 EASE2_S36km.lats.500x500x1.double

0000000    -9,990000000000000e+02   -9,990000000000000e+02
*
0000080

This causes the NCL script ouput netCDF file to also have missing values in the latitude and longitude
dimension variables which is not allowed. Certainly this is also the reason why python runs on an error.

ncdump -v lon EASE2_S36km.lats.500x500_grid.nc

netcdf EASE2_S36km.lats.500x500_grid {
dimensions:
    time = UNLIMITED ; // (1 currently)
    lat = 500 ;
    lon = 500 ;
variables:
    double time(time) ;
        time:long_name = "time" ;
        time:calendar = "standard" ;
        time:units = "hours since 2020-01-01 00:00:00" ;
    double lat(lat) ;
        lat:units = "degrees_north" ;
        lat:long_name = "latitude" ;
    double lon(lon) ;
        lon:units = "degrees_east" ;
        lon:long_name = "longitude" ;
    float data(time, lat, lon) ;
        data :_FillValue = 9.96921e+36f ;

// global attributes:
        :creation_date = "Mi  1 Jul 2020 11:23:26 CEST" ;
        :source_file = "EASE2_S36km.lats.500x500x1.double; EASE2_S36km.lons.500x500x1.double" ;
        :title = "Convert binary to netCDF" ;
data:

 lon = -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, 
    -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, 
    -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, 
    -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, 
    -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, 
    -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, 
    -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, 
    -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, 
    -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, 
    -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, 
    -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, 
    -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, 
    -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, 
    -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, 
    -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, 
    -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, 
    -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, 
    -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, 
    -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, 
    -999, -999, -999, -4.46894225455147, -4.24062420626405, 
    -4.01217115496777, -3.78359025109984, -3.55488866938136, 
    -3.32607360749208, -3.09715228473433, -2.86813194068669, 
    -2.63901983384809, -2.40982324027282, -2.18054945219727, 
    -1.95120577665882, -1.72179953410781, -1.49233805701292, 
    -1.26282868846094, -1.03327878075134, -0.803695693986433, 
    -0.574086794657846, -0.344459454229829, -0.114821047720236, 
    0.114821047720236, 0.344459454229829, 0.574086794657846, 
    0.803695693986433, 1.03327878075134, 1.26282868846094, 1.49233805701292, 
    1.72179953410781, 1.95120577665882, 2.18054945219727, 2.40982324027282, 
    2.63901983384809, 2.86813194068669, 3.09715228473433, 3.32607360749208, 
    3.55488866938136, 3.78359025109984, 4.01217115496777, 4.24062420626405, 
    4.46894225455147, -999, -999, -999, -999, -999, -999, -999, -999, -999, 
    -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, 
    -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, 
    -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, 
    -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, 
    -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, 
    -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, 
    -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, 
    -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, 
    -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, 
    -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, 
    -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, 
    -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, 
    -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, 
    -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, 
    -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, 
    -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, 
    -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, 
    -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, 
    -999, -999, -999, -999, -999 ;
}

ncdump -v lat EASE2_S36km.lats.500x500_grid.nc

netcdf EASE2_S36km.lats.500x500_grid {
dimensions:
    time = UNLIMITED ; // (1 currently)
    lat = 500 ;
    lon = 500 ;
variables:
    double time(time) ;
        time:long_name = "time" ;
        time:calendar = "standard" ;
        time:units = "hours since 2020-01-01 00:00:00" ;
    double lat(lat) ;
        lat:units = "degrees_north" ;
        lat:long_name = "latitude" ;
    double lon(lon) ;
        lon:units = "degrees_east" ;
        lon:long_name = "longitude" ;
    float data(time, lat, lon) ;
        data :_FillValue = 9.96921e+36f ;

// global attributes:
        :creation_date = "Mi  1 Jul 2020 11:23:26 CEST" ;
        :source_file = "EASE2_S36km.lats.500x500x1.double; EASE2_S36km.lons.500x500x1.double" ;
        :title = "Convert binary to netCDF" ;
data:

 lat = -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, 
    -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, 
    -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, 
    -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, 
    -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, 
    -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, 
    -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, 
    -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, 
    -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, 
    -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, 
    -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, 
    -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, 
    -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, 
    -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, 
    -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, 
    -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, 
    -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, 
    -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, 
    -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, 
    -999, -999, -999, -0.00732852404086119, -0.0422434159069207, 
    -0.0753206957637075, -0.106560371212084, -0.135962446694351, 
    -0.163526923962869, -0.189253802522153, -0.213143080042346, 
    -0.235194752745714, -0.255408815765307, -0.273785263476192, 
    -0.290324089798837, -0.305025288475207, -0.317888853316978, 
    -0.328914778426491, -0.338103058389853, -0.345453688442348, 
    -0.350966664606736, -0.35464198380346, -0.356479643933511, 
    -0.356479643933511, -0.35464198380346, -0.350966664606736, 
    -0.345453688442348, -0.338103058389853, -0.328914778426491, 
    -0.317888853316978, -0.305025288475207, -0.290324089798837, 
    -0.273785263476192, -0.255408815765307, -0.235194752745714, 
    -0.213143080042346, -0.189253802522153, -0.163526923962869, 
    -0.135962446694351, -0.106560371212084, -0.0753206957637075, 
    -0.0422434159069207, -0.00732852404086119, -999, -999, -999, -999, -999, 
    -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, 
    -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, 
    -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, 
    -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, 
    -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, 
    -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, 
    -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, 
    -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, 
    -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, 
    -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, 
    -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, 
    -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, 
    -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, 
    -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, 
    -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, 
    -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, 
    -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, 
    -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, 
    -999, -999, -999, -999, -999, -999, -999, -999, -999 ;
}

RE: Remapbil with lat/lon binary files - Added by Sentia Goursaud almost 4 years ago

Dear Karin,

Thanks for the response.

Well I guess I have to find another solution to switch my simulations to an equal area grid. Do you know if I can do it with cdo without a target grid ?

Thank you very much for your help,
Sentia

    (1-8/8)