Project

General

Profile

Problem converting a generic nc grid to a latlon nc grid

Added by Marc Chiacchio almost 9 years ago

Hello,

I have seen the same problems with CDO reading a generic grid netcdf file but nothing seems to work so far. In my case, I just want to convert this generic grid to a regular latlon grid so that I can use sellonlatbox. I have used the setgrid function and it seems to work but the resulting file is not correct. It seems the data values are not put back correctly in the same locations as before. The original netcdf file is not global and instead is a box centered over the ocean with edges that contour around the coasts. This may be the problem but I'm not sure. If I do sinfo on this original file I get this:

File format: netCDF
-1 : Institut Source Ttype Levels Num Gridsize Num Dtype : Parameter ID
1 : unknown unknown constant 344 1 1650 1 F32 : -1
2 : unknown unknown constant 1 2 344 2 F32 : -2
3 : unknown unknown constant 1 2 30 3 F32 : -3
4 : unknown unknown constant 1 2 55 4 F32 : -4
Grid coordinates :
1 : generic > size : dim = 1650 nx = 30 ny = 55
2 : generic > size : dim = 344 nx = 344 ny = 0
3 : generic > size : dim = 30 nx = 30 ny = 0
4 : generic > size : dim = 55 nx = 55 ny = 0
Vertical coordinates :
1 : generic level : 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19
20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35
36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51
52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67
68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83
84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99
100 101 102 103 104 105 106 107 108 109 110 111
112 113 114 115 116 117 118 119 120 121 122 123
124 125 126 127 128 129 130 131 132 133 134 135
136 137 138 139 140 141 142 143 144 145 146 147
148 149 150 151 152 153 154 155 156 157 158 159
160 161 162 163 164 165 166 167 168 169 170 171
172 173 174 175 176 177 178 179 180 181 182 183
184 185 186 187 188 189 190 191 192 193 194 195
196 197 198 199 200 201 202 203 204 205 206 207
208 209 210 211 212 213 214 215 216 217 218 219
220 221 222 223 224 225 226 227 228 229 230 231
232 233 234 235 236 237 238 239 240 241 242 243
244 245 246 247 248 249 250 251 252 253 254 255
256 257 258 259 260 261 262 263 264 265 266 267
268 269 270 271 272 273 274 275 276 277 278 279
280 281 282 283 284 285 286 287 288 289 290 291
292 293 294 295 296 297 298 299 300 301 302 303
304 305 306 307 308 309 310 311 312 313 314 315
316 317 318 319 320 321 322 323 324 325 326 327
328 329 330 331 332 333 334 335 336 337 338 339
340 341 342 343 344
2 : surface : 0

With help from the other posts here, I have used the ncrename NCO function to rename the dimension t to time but it's still giving me the same problem as before.
If I do griddes on this same original file it looks like this:

#
  1. gridID 2 #
    gridtype = generic
    gridsize = 1650
    xname = x
    xunits = degrees
    yname = y
    yunits = degrees
    xsize = 30
    ysize = 55 #
  2. gridID 3 #
    gridtype = generic
    gridsize = 344
    xname = x
    xunits = degrees
    yname = y
    yunits = degrees
    xsize = 344 #
  3. gridID 4 #
    gridtype = generic
    gridsize = 30
    xname = x
    xunits = degrees
    yname = y
    yunits = degrees
    xsize = 30 #
  4. gridID 5 #
    gridtype = generic
    gridsize = 55
    xname = x
    xunits = degrees
    yname = y
    yunits = degrees
    xsize = 55

I'm not familiar as to why there are four grids associated with this file but once I use the setgrid function, the resulting output file using griddes has this information:

#
  1. gridID 2 #
    gridtype = lonlat
    gridsize = 1650
    xname = lon
    xlongname = longitude
    xunits = degrees_east
    yname = lat
    ylongname = latitude
    yunits = degrees_north
    xsize = 55
    ysize = 30
    xfirst = -64.5
    xinc = 1
    yfirst = 29.5
    yinc = -1 #
  2. gridID 3 #
    gridtype = generic
    gridsize = 30
    xname = x
    xunits = degrees
    yname = y
    yunits = degrees
    xsize = 30 #
  3. gridID 4 #
    gridtype = generic
    gridsize = 55
    xname = x
    xunits = degrees
    yname = y
    yunits = degrees
    xsize = 55

and the output file is still not correct even though it looks like it converted it to a latlon grid. Did anyone ever have similar problems?


Replies (7)

RE: Problem converting a generic nc grid to a latlon nc grid - Added by Nicole C Laureanti over 2 years ago

Marc Chiacchio wrote:

Hello,

I have seen the same problems with CDO reading a generic grid netcdf file but nothing seems to work so far. In my case, I just want to convert this generic grid to a regular latlon grid so that I can use sellonlatbox. I have used the setgrid function and it seems to work but the resulting file is not correct. It seems the data values are not put back correctly in the same locations as before. The original netcdf file is not global and instead is a box centered over the ocean with edges that contour around the coasts. This may be the problem but I'm not sure. If I do sinfo on this original file I get this:

File format: netCDF
-1 : Institut Source Ttype Levels Num Gridsize Num Dtype : Parameter ID
1 : unknown unknown constant 344 1 1650 1 F32 : -1
2 : unknown unknown constant 1 2 344 2 F32 : -2
3 : unknown unknown constant 1 2 30 3 F32 : -3
4 : unknown unknown constant 1 2 55 4 F32 : -4
Grid coordinates :
1 : generic > size : dim = 1650 nx = 30 ny = 55
2 : generic > size : dim = 344 nx = 344 ny = 0
3 : generic > size : dim = 30 nx = 30 ny = 0
4 : generic > size : dim = 55 nx = 55 ny = 0
Vertical coordinates :
1 : generic level : 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19
20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35
36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51
52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67
68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83
84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99
100 101 102 103 104 105 106 107 108 109 110 111
112 113 114 115 116 117 118 119 120 121 122 123
124 125 126 127 128 129 130 131 132 133 134 135
136 137 138 139 140 141 142 143 144 145 146 147
148 149 150 151 152 153 154 155 156 157 158 159
160 161 162 163 164 165 166 167 168 169 170 171
172 173 174 175 176 177 178 179 180 181 182 183
184 185 186 187 188 189 190 191 192 193 194 195
196 197 198 199 200 201 202 203 204 205 206 207
208 209 210 211 212 213 214 215 216 217 218 219
220 221 222 223 224 225 226 227 228 229 230 231
232 233 234 235 236 237 238 239 240 241 242 243
244 245 246 247 248 249 250 251 252 253 254 255
256 257 258 259 260 261 262 263 264 265 266 267
268 269 270 271 272 273 274 275 276 277 278 279
280 281 282 283 284 285 286 287 288 289 290 291
292 293 294 295 296 297 298 299 300 301 302 303
304 305 306 307 308 309 310 311 312 313 314 315
316 317 318 319 320 321 322 323 324 325 326 327
328 329 330 331 332 333 334 335 336 337 338 339
340 341 342 343 344
2 : surface : 0

With help from the other posts here, I have used the ncrename NCO function to rename the dimension t to time but it's still giving me the same problem as before.
If I do griddes on this same original file it looks like this:

#
  1. gridID 2 #
    gridtype = generic
    gridsize = 1650
    xname = x
    xunits = degrees
    yname = y
    yunits = degrees
    xsize = 30
    ysize = 55 #
  2. gridID 3 #
    gridtype = generic
    gridsize = 344
    xname = x
    xunits = degrees
    yname = y
    yunits = degrees
    xsize = 344 #
  3. gridID 4 #
    gridtype = generic
    gridsize = 30
    xname = x
    xunits = degrees
    yname = y
    yunits = degrees
    xsize = 30 #
  4. gridID 5 #
    gridtype = generic
    gridsize = 55
    xname = x
    xunits = degrees
    yname = y
    yunits = degrees
    xsize = 55

I'm not familiar as to why there are four grids associated with this file but once I use the setgrid function, the resulting output file using griddes has this information:

#
  1. gridID 2 #
    gridtype = lonlat
    gridsize = 1650
    xname = lon
    xlongname = longitude
    xunits = degrees_east
    yname = lat
    ylongname = latitude
    yunits = degrees_north
    xsize = 55
    ysize = 30
    xfirst = -64.5
    xinc = 1
    yfirst = 29.5
    yinc = -1 #
  2. gridID 3 #
    gridtype = generic
    gridsize = 30
    xname = x
    xunits = degrees
    yname = y
    yunits = degrees
    xsize = 30 #
  3. gridID 4 #
    gridtype = generic
    gridsize = 55
    xname = x
    xunits = degrees
    yname = y
    yunits = degrees
    xsize = 55

and the output file is still not correct even though it looks like it converted it to a latlon grid. Did anyone ever have similar problems?

Hello Marc,
I'm having the same issue. Did you manage to solve by yourself? Could you kindly give me some advice?

RE: Problem converting a generic nc grid to a latlon nc grid - Added by Karin Meier-Fleischer over 2 years ago

Hi Nicole,

without the data you want to remap it is really hard to help. Can you provide data (1 timestep is enough)?

I guess that the missing units and wrong coordinate names (y, x instead of lat, lon) are causing the problem. But to verify it it would be good to have the data or at least the output of ncdump -h you_data_file.

And just to make it more clear, in the above example the input file seems to contain one variable -1(time,level,x,y) and variables -2 to -4 with dimensions (time,y,x). Btw, it is always a good idea to use cdo sinfon to display the real variable names, too. With setgrid you choose the grid on which the operation will be done, otherwise it will give you an error or warning.

-Karin

RE: Problem converting a generic nc grid to a latlon nc grid - Added by Nicole C Laureanti over 2 years ago

Hi Karin,

Thanks for your answer.

My intention is to use the data from "ocean_month_z_0036_01.nc" and adapt for "ocean_hgrid.nc" grid. Whenever I use remapbil it gives me the error

cdo remapbil (Abort): Unsupported target grid type (generic)!

The output for the data is:

netcdf ocean_month_z_0036_01 {
dimensions:
xq = 360 ;
yh = 292 ;
z_l = 34 ;
z_i = 35 ;
Time = UNLIMITED ; // (1 currently)
nv = 2 ;
xh = 360 ;
yq = 292 ;
variables:
double xq(xq) ;
xq:long_name = "q point nominal longitude" ;
xq:units = "degrees_E" ;
xq:cartesian_axis = "X" ;
double yh(yh) ;
yh:long_name = "h point nominal latitude" ;
yh:units = "degrees_N" ;
yh:cartesian_axis = "Y" ;
double z_l(z_l) ;
z_l:long_name = "Depth at cell center" ;
z_l:units = "meters" ;
z_l:cartesian_axis = "Z" ;
z_l:positive = "down" ;
z_l:edges = "z_i" ;
double z_i(z_i) ;
z_i:long_name = "Depth at interface" ;
z_i:units = "meters" ;
z_i:cartesian_axis = "Z" ;
z_i:positive = "down" ;
double Time(Time) ;
Time:long_name = "Time" ;
Time:units = "days since 0001-01-01 00:00:00" ;
Time:cartesian_axis = "T" ;
Time:calendar_type = "NOLEAP" ;
Time:calendar = "NOLEAP" ;
Time:bounds = "Time_bnds" ;
double nv(nv) ;
nv:long_name = "vertex number" ;
nv:units = "none" ;
nv:cartesian_axis = "N" ;
double xh(xh) ;
xh:long_name = "h point nominal longitude" ;
xh:units = "degrees_E" ;
xh:cartesian_axis = "X" ;
double yq(yq) ;
yq:long_name = "q point nominal latitude" ;
yq:units = "degrees_N" ;
yq:cartesian_axis = "Y" ;
float u(Time, z_l, yh, xq) ;
u:long_name = "Zonal velocity" ;
u:units = "meter second-1" ;
u:missing_value = -1.e+34f ;
u:_FillValue = -1.e+34f ;
u:cell_methods = "z_l:mean yh:mean xq:point time: mean" ;
u:time_avg_info = "average_T1,average_T2,average_DT" ;
u:interp_method = "none" ;
float v(Time, z_l, yq, xh) ;
v:long_name = "Meridional velocity" ;
v:units = "meter second-1" ;
v:missing_value = -1.e+34f ;
v:_FillValue = -1.e+34f ;
v:cell_methods = "z_l:mean yq:point xh:mean time: mean" ;
v:time_avg_info = "average_T1,average_T2,average_DT" ;
v:interp_method = "none" ;
float temp(Time, z_l, yh, xh) ;
temp:long_name = "Potential Temperature" ;
temp:units = "Celsius" ;
temp:missing_value = -1.e+34f ;
temp:_FillValue = -1.e+34f ;
temp:cell_methods = "z_l:mean yh:mean xh:mean time: mean" ;
temp:time_avg_info = "average_T1,average_T2,average_DT" ;
float salt(Time, z_l, yh, xh) ;
salt:long_name = "Salinity" ;
salt:units = "PPT" ;
salt:missing_value = -1.e+34f ;
salt:_FillValue = -1.e+34f ;
salt:cell_methods = "z_l:mean yh:mean xh:mean time: mean" ;
salt:time_avg_info = "average_T1,average_T2,average_DT" ;
double average_T1(Time) ;
average_T1:long_name = "Start time for average period" ;
average_T1:units = "days since 0001-01-01 00:00:00" ;
average_T1:missing_value = 1.e+20 ;
average_T1:_FillValue = 1.e+20 ;
double average_T2(Time) ;
average_T2:long_name = "End time for average period" ;
average_T2:units = "days since 0001-01-01 00:00:00" ;
average_T2:missing_value = 1.e+20 ;
average_T2:_FillValue = 1.e+20 ;
double average_DT(Time) ;
average_DT:long_name = "Length of average period" ;
average_DT:units = "days" ;
average_DT:missing_value = 1.e+20 ;
average_DT:_FillValue = 1.e+20 ;
double Time_bnds(Time, nv) ;
Time_bnds:long_name = "Time axis boundaries" ;
Time_bnds:units = "days" ;
Time_bnds:missing_value = 1.e+20 ;
Time_bnds:_FillValue = 1.e+20 ;

// global attributes:
:filename = "00330101.ocean_month_z_0036_01.nc" ;
:title = "GOLD_SIS" ;
:grid_type = "regular" ;
:grid_tile = "N/A" ;
}

cdo sinfon ocean_month_z_0036_01.nc

Warning (find_time_vars): Found more than one time variable, skipped variable average_T1!
Warning (find_time_vars): Found more than one time variable, skipped variable average_T2!
File format : NetCDF2
-1 : Institut Source T Steptype Levels Num Points Num Dtype : Parameter name
1 : unknown unknown v instant 34 1 105120 1 F32 : u
2 : unknown unknown v instant 34 1 105120 2 F32 : v
3 : unknown unknown v instant 34 1 105120 3 F32 : temp
4 : unknown unknown v instant 34 1 105120 3 F32 : salt
5 : unknown unknown v instant 1 2 1 4 F64 : average_DT
Grid coordinates :
1 : lonlat : points=105120 (360x292)
xq : -279 to 80 by 1 degrees_E circular
yh : -74.75 to 89.75 degrees_N
2 : lonlat : points=105120 (360x292)
xh : -279.5 to 79.5 by 1 degrees_E circular
yq : -74.49961 to 90 degrees_N
3 : lonlat : points=105120 (360x292)
xh : -279.5 to 79.5 by 1 degrees_E circular
yh : -74.75 to 89.75 degrees_N
4 : generic : points=1
Vertical coordinates :
1 : generic : levels=34
z_l : 2.5 to 6000 meters
2 : surface : levels=1
Time coordinate : 1 step
RefTime = 0001-01-01 00:00:00 Units = days Calendar = 365_day Bounds = true
YYYY-MM-DD hh:mm:ss YYYY-MM-DD hh:mm:ss YYYY-MM-DD hh:mm:ss YYYY-MM-DD hh:mm:ss
0036-01-16 12:00:00

The grid information is what follows:

netcdf ocean_hgrid {
dimensions:
string = 255 ;
nx = 320 ;
ny = 160 ;
nxp = 321 ;
nyp = 161 ;
variables:
char tile(string) ;
tile:standard_name = "grid_tile_spec" ;
tile:geometry = "spherical" ;
tile:north_pole = "0.0 90.0" ;
tile:discretization = "logically_rectangular" ;
tile:conformal = "true" ;
double x(nyp, nxp) ;
x:standard_name = "geographic_longitude" ;
x:units = "degree_east" ;
x:coordinates = "lat lon" ;
double y(nyp, nxp) ;
y:standard_name = "geographic_latitude" ;
y:units = "degree_north" ;
double dx(nyp, nx) ;
dx:standard_name = "grid_edge_x_distance" ;
dx:units = "meters" ;
double dy(ny, nxp) ;
dy:standard_name = "grid_edge_y_distance" ;
dy:units = "meters" ;
double area(ny, nx) ;
area:standard_name = "grid_cell_area" ;
area:units = "m2" ;
double angle_dx(nyp, nxp) ;
angle_dx:standard_name = "grid_vertex_x_angle_WRT_geographic_east" ;
angle_dx:units = "degrees_east" ;
char arcx(string) ;
arcx:standard_name = "grid_edge_x_arc_type" ;
arcx:north_pole = "0.0 90.0" ;

// global attributes:
:grid_version = "0.2" ;
:code_version = "$Name: fre-nctools-bronx-10 $" ;
:history = "Fri Dec 10 19:15:12 2021: ncatted -a coordinates,x,c,c,y x /home/nlaureanti/Documentos/INPE/mom/grid_regionalFRE/emanuel_dec21/ocean_hgrid.nc\n",
"Fri Dec 10 19:14:46 2021: ncatted -a coordinates,x,c,c,y x /home/nlaureanti/Documentos/INPE/mom/grid_regionalFRE/emanuel_dec21/ocean_hgrid.nc\n",
"Fri Dec 10 17:37:36 2021: ncatted -a coordinates,x,c,c,lat lon /home/nlaureanti/Documentos/INPE/mom/grid_regionalFRE/emanuel_dec21/ocean_hgrid.nc\n",
"make_hgrid --grid_type regular_lonlat_grid --nxbnd 2 --nybnd 2 --xbnd -60,-20 --ybnd -40,-20 --dlon 0.25,0.25 --dlat 0.25,0.25 --grid_name ocean_hgrid --center c_cell" ;
:NCO = "4.7.2" ;
}

Warning (cdf_scan_var_attr): NetCDF: Variable not found - >lat<
Warning (cdf_scan_var_attr): NetCDF: Variable not found - >lon< #
  1. gridID 1 #
    gridtype = generic
    gridsize = 51681
    xsize = 321
    ysize = 161
    scanningMode = 64 #
  2. gridID 2 #
    gridtype = generic
    gridsize = 51520
    xsize = 320
    ysize = 161
    scanningMode = 64 #
  3. gridID 3 #
    gridtype = generic
    gridsize = 51360
    xsize = 321
    ysize = 160
    scanningMode = 64 #
  4. gridID 4 #
    gridtype = generic
    gridsize = 51200
    xsize = 320
    ysize = 160
    scanningMode = 64

Warning (cdf_scan_var_attr): NetCDF: Variable not found - >lat<
Warning (cdf_scan_var_attr): NetCDF: Variable not found - >lon<
File format : NetCDF4 classic
-1 : Institut Source T Steptype Levels Num Points Num Dtype : Parameter name
1 : unknown unknown c instant 1 1 51681 1 F64 : x
2 : unknown unknown c instant 1 1 51681 1 F64 : y
3 : unknown unknown c instant 1 1 51520 2 F64 : dx
4 : unknown unknown c instant 1 1 51360 3 F64 : dy
5 : unknown unknown c instant 1 1 51200 4 F64 : area
6 : unknown unknown c instant 1 1 51681 1 F64 : angle_dx
Grid coordinates :
1 : generic : points=51681 (321x161)
2 : generic : points=51520 (320x161)
3 : generic : points=51360 (321x160)
4 : generic : points=51200 (320x160)
Vertical coordinates :
1 : surface : levels=1

RE: Problem converting a generic nc grid to a latlon nc grid - Added by Karin Meier-Fleischer over 2 years ago

Neither your input file nor your grid file contains the coordinate variables lon (or longitude) or lat (or latitude). CDO does not kniow hoe to handle all these different grids. The best would be to extract the variable you want to remap and add the coordinates attribute to the variable that points to the grid x and y. I'm not sure, maybe you have to rename the coordinate variables to lon and lat, too. Can you upload one timestep of your input data and the grid file?

RE: Problem converting a generic nc grid to a latlon nc grid - Added by Nicole C Laureanti over 2 years ago

I believe I never did any of those steps you suggested. You will find the files attached.

RE: Problem converting a generic nc grid to a latlon nc grid - Added by Karin Meier-Fleischer over 2 years ago

After adding the coordinates attribute I was able to remap the data to a global 1 degree grid (r360x180) for instance.

cdo -setattribute,temp@coordinates="yh xh" -selname,temp $infile infile2.nc
cdo remapbil,r360x180 infile2.nc outfile_1deg.nc

Result:

cdo sinfon outfile_1deg.nc
   File format : NetCDF2
    -1 : Institut Source   T Steptype Levels Num    Points Num Dtype : Parameter name
     1 : unknown  unknown  v instant      34   1     64800   1  F32  : temp          
   Grid coordinates :
     1 : lonlat                   : points=64800 (360x180)
                              lon : 0 to 359 by 1 degrees_east  circular
                              lat : -89.5 to 89.5 by 1 degrees_north
   Vertical coordinates :
     1 : generic                  : levels=34
                              z_l : 2.5 to 6000 meters
   Time coordinate :  1 step
     RefTime =  0001-01-01 00:00:00  Units = days  Calendar = 365_day  Bounds = true
  YYYY-MM-DD hh:mm:ss  YYYY-MM-DD hh:mm:ss  YYYY-MM-DD hh:mm:ss  YYYY-MM-DD hh:mm:ss
  0036-01-16 12:00:00

The target gridfile (curvilinear) is not correct. The variables x and y are not dimension variables.

RE: Problem converting a generic nc grid to a latlon nc grid - Added by Nicole C Laureanti over 2 years ago

Hi Karin, for now, just to remap the data file helps me a lot. I see that both files are written over incompatible C-Arakawa grids and was hoping that cdo could adjust this. I'll try to continue this through python or convert it to another latlon grid. Thanks for your help anyway, you're great!

    (1-7/7)