CDO

Your data keep an extra column on the left and right side. These contain a replication of the second and second last column. This hides the cyclic interpolation over the edges. A similar problem concerns the first and last row.
With selindexbox these columns and rows can be removed:

cdo remapbil,r360x180  -selindexbox,2,361,2,293

Cheers,
Uwe

That data is on the tripolar eORCA1 grid. The gaps you see are over the seams in the grid. It looks like CDO does not know how to join the seams. You might have to manually specify your input grid specifications in the CDO-SCRIP format. Read the CDO manual sections on interpolation. You need to specify the grid corners as well as the grid centers which will require the full eORCA1 grid. The full grid is actually 4 grids as there are separate grids for U,V,T and f-points (Arakawa-C grid) which are offset from each other by a 1/2 grid. Exploit the offsets to get the corner points for each grid e.g. the corners of a T-point grid cell are the surrounding f-point grid cells.

You could also use the grid in the data file (which is just the T-point grid) and somehow generate your own corners by interpolation and extrapolation. This is what CDO is attempting to do in your example, with partial success.

Your best option is to ask the data provider if/when they are going to eventually provide the same data interpolated to a 1x1 degree regular grid. This is a CMIP6 requirement for ocean models on curvilinear grids. These files will have identical names but the _gn_ in the file name will be _gr_ or _grX_ where X is a number.

Welp, after reading Uwe's reply, please ignore my post.

Dear Uwe and Brendan,

thanks for the reply. Both suggesions help.

@Uwe One more question: How can you know there are replications in the data? Did you plot the data, or just use some nc tools to check the data?
I'm asking this because there are some other CMIP6 models which show the same missing-value-line problem after remapped by cdo. So I need to know which column or row are replicated.

Many thanks,
Shizhu

@Uwe

Like this enclosed file (zos_Omon_CanESM5_ssp119_r10i1p1f1_gn_201501.nc)

cdo remapbil

If CDO detects a cyclic grid, the term circular appears in the grid section of sinfon at the longitudes.
Here is an example:

cdo sinfon -selindexbox,2,361,2,293 zos_Omon_CNRM-CM6-1_abrupt-4xCO2_r5i1p1f2_gn_185009-185912.nc
result:
   ...
   Grid coordinates :
     1 : curvilinear              : points=105120 (360x292)
                              lon : -179.9965 to 179.9903 degrees_east  circular
                              lat : -78.79526 to 89.74177 degrees_north
   ...

We use the undocumented CDO operator verifygrid to verify the grid information of a file. Here is the output for the above file:

cdo verifygrid zos_Omon_CNRM-CM6-1_abrupt-4xCO2_r5i1p1f2_gn_185009-185912.nc
cdo    verifygrid: Grid consists of 106428 cells (type: curvilinear), of which
cdo    verifygrid:        30 cells have 3 vertices
cdo    verifygrid:    106036 cells have 4 vertices
cdo    verifygrid:        30 cells have duplicate vertices
cdo    verifygrid:       362 cells have unusable vertices
cdo    verifygrid:       947 cells are not unique
cdo    verifygrid:       369 cells are non-convex
cdo    verifygrid:         9 cells have their vertices arranged in a clockwise order
cdo    verifygrid:       153 cells have their center points located outside their boundaries

Uwe. That command is gold.

Shizhu. It is a requirement of CMIP6 that all ocean variables are provided on the native grid, with vector components relative to the grid i.e. _gn_ in the file name means native grid. Since a lot of the ocean models in CMIP6 are using the eORCA1 grid, they might all have this feature of redundant rows and columns. Do an ncdump -h and scan the global attributes to see what the ocean grid is and/or see this page https://wcrp-cmip.github.io/CMIP6_CVs/docs/CMIP6_source_id.html which generated from the CMIP6_source_id.json file here: https://github.com/WCRP-CMIP/CMIP6_CVs. There is also the ES-DOC explorer site but it is not very complete yet: https://explore.es-doc.org/

@Uwe

I have tried the sinfo and sinfon with the enclosed file (zos_Omon_IPSL-CM6A-LR_dcppC-amv-ExTrop-pos_r13i1p1f1_gn_1850.nc).

The following command gives the circular term:

(base) swang@myUbt:~/test/cdo/IPSL-CM6A-LR$ cdo sinfo -selindexbox,1,361,1,332 zos_Omon_IPSL-CM6A-LR_dcppC-amv-ExTrop-pos_r13i1p1f1_gn_1850.nc
   File format : NetCDF4
   ...        
   Grid coordinates :
     1 : curvilinear              : points=119852 (361x332)
                          nav_lon : -179.9965 to 179.9903 degrees_east  circular
                          nav_lat : -85.78874 to 89.74177 degrees_north
                        available : cellbounds
   ...

cdo remapbil,r360x180 -selindexbox,1,361,1,332 zos_Omon_IPSL-CM6A-LR_dcppC-amv-ExTrop-pos_r13i1p1f1_gn_1850.nc aa.nc

On the other hand, the following command also shows the circular term and did a remapping without the missing-value line.

(base) swang@myUbt:~/test/cdo/IPSL-CM6A-LR$ cdo sinfo -selindexbox,2,361,2,331 zos_Omon_IPSL-CM6A-LR_dcppC-amv-ExTrop-pos_r13i1p1f1_gn_1850.nc
cdo sinfo: Started child process "selindexbox,2,361,2,331 zos_Omon_IPSL-CM6A-LR_dcppC-amv-ExTrop-pos_r13i1p1f1_gn_1850.nc (pipe1.1)".
   File format : NetCDF4
   ...
     1 : curvilinear              : points=118800 (360x330)
                          nav_lon : -179.9965 to 179.9903 degrees_east  circular
                          nav_lat : -85.63117 to 89.74177 degrees_north
                        available : cellbounds
   ...

cdo remapbil,r360x180 -selindexbox,2,361,2,331 zos_Omon_IPSL-CM6A-LR_dcppC-amv-ExTrop-pos_r13i1p1f1_gn_1850.nc bb.nc

The verifygrid operator shows us the general infomation of the grid. Is it possible to know exactly which column or row is replicated? My choice would be Python or Matlab. Is there any nc tools which I can rely on neatly?
For example, for the enclosed file zos_Omon_CanESM5_ssp119_r10i1p1f1_gn_201501.nc
I tried the following indexboxes but no one gives me the correct remapping.

cdo remapbil,r360x180 -selindexbox,1,360,1,291 zos_Omon_CanESM5_ssp119_r10i1p1f1_gn_201501.nc aa.nc
cdo remapbil,r360x180 -selindexbox,1,360,2,290 zos_Omon_CanESM5_ssp119_r10i1p1f1_gn_201501.nc aa.nc
cdo remapbil,r360x180 -selindexbox,2,359,1,291 zos_Omon_CanESM5_ssp119_r10i1p1f1_gn_201501.nc aa.nc

@Brendan DeTracey
Yes you are right. CMIP6 requires ocean data to be archived on the model native grid (https://docs.google.com/document/d/1kZw3KXvhRAJdBrXHhXo4f6PDl_NzrFre1UfWGHISPz4/edit).
I checked the CIMP6 models whose oceans are on ORCA or eORCA grid. Then I use cdo remapbil to remap the raw data and found that cdo can remap some models correctly and the others wrongly (see the enlosed file CMIP6_CDO_REMAP.docx).
So what we can confirm is that not all the ORCA-based ocean results have the feature of redundant rows and columns. I guess some model results are providing the grid information (e.g., the cell_bounds) which cdo can not work with correctly.

@Shizhu
Thanks for the information about the CMIP6 ocean files. It is good to know that some of them have this problem. Looks like the CMIP6 team needs another requirement for ocean grids i.e. do not include redundant rows and columns. I was hoping CMIP6 would resolve these inconsistencies. What a nuisance.

Thats right, the check for the circular term is very simple and not perfect. So it is not guarantee that this will work correctly, sorry.
I seems it works correctly with -selindexbox,2,362,1,332.
verifygrid prints some more information with the CDO option -v. The output is useful for me but not perfect. I will try to optimize the code for the next CDO release. Here is an example:

cdo -v verifygrid zos_Omon_CanESM5_ssp119_r10i1p1f1_gn_201501.nc
...
The duplicate vertex -0.118918, -0.388964, 0.913545 was found in cell no 104566.
The duplicate vertex -0.118918, -0.388964, 0.913545 was found in cell no 104567.
The duplicate vertex -0.118918, -0.388964, 0.913545 was found in cell no 104568.
...

I'm currently also into remapping of CMIP6 Omon data onto a regular lon/lat grid and appreciate the hints and findings already provided.

Please allow me to ask an interim question regarding the vector component variables uo and vo. How are they oriented to? Relative to the true poles or to the poles of the underlying ocean grid. Is there a general answer for all OGCMs or how much does it depend on the specifics of the ocean model? I'm currently looking at UKESM1-0-LL, MPI-ESM1-2-HR, GFDL-ESM4 and IPSL-CM6A-LR.

Best,
Matthias

since these model use their own grids + discretizations there won't a common solution. You have to ask the model owners, or check the long names. Those sometimes contain hints like westward wind speed@ or so.

If grid descriptor is native grid, *_gn_*.nc, they should be relative to the grid. This should also be reflected in the variable description attribute, where it should state "Prognostic x-ward velocity component resolved by the model.". X-ward and y-ward for native direction, meridional and zonal for geographic direction.

Thanks you both, that helped a lot.

However, uo+vo from all 4 models only provide the native grid and I'm far away from getting this solved on my own.

for MPIOM (ocean component of MPI_ESM) you can use the CDO operator mrotuvb. this will create N/E vector components for the transformed vector at the cell centre. For ECHAM (MPI-ESM atm) the wind vector already are as you want it.

hth
ralf

Thanks, is mrotuvb documented somewhere? Can't find it. Neither in latest PDF nor in HTML documentation.

What is expected for the second input stream?

My guess? First file is u file, second file is v file, and rotated u and v are both written to the output file. FYI, I just noticed that the python library Iris (https://scitools.org.uk/iris/docs/latest/) has this functionality (https://scitools.org.uk/iris/docs/latest/iris/iris/analysis/cartography.html#iris.analysis.cartography.rotate_grid_vectors). I have never used Iris myself...yet. ESMValTool and ESMValCore are built using Iris but a quick grep of their sources for cartography.rotate_grid_vectors shows it hasn't been used anywhere. This is a little troubling to me.

I use cdo remapbil,r360x180 to remap the following two files.
uo_Omon_UKESM1-0-LL_ssp370_r3i1p1f2_gn_210001-210012.nc
vo_Omon_UKESM1-0-LL_ssp370_r3i1p1f2_gn_210001-210012.nc

Here is the output of the surface velocity.

At least for me, the big pattern makes sense. This means the uo/vo on the UKESM1-0-LL native grid is in the east/north direction.
My suggestion is you use cdo to remap the uo/vo directly and check the output. This might be the easiest way.
Sincerely, I really hope every participant group of CMIP6 can provide their own gird weight files.

With respect to the mrotuvb operator, you can check the slide by Michael Böttinger from DKRZ (https://www.dkrz.de/mms/pdf/vis/Tut_VectorVisualization.pdf).

Cheers,
Shizhu

See this page describing the ORCA tri-polar grid (https://www.nemo-ocean.eu/doc/node108.html). The native grid directions deviate from geographic, especially in the Arctic and North Altantic.

And I would be careful of the procedure in that pdf which does not seem suitable to our case. Mrotuvb assumes your data is on a Arakawa C grid and interpolates u and v to a common location (C grid mass point) before rotating.

Hey, in case you haven't found a solution or somebody else is struggling with this, try to use remapdis instead of remapbil. I tried everything in the world, but for me using remapbil fixed this

thank you sooooooooooo much!

Marlen Kolbe wrote in RE: cdo remapbil creates missing values when regridding C...:

Hey, in case you haven't found a solution or somebody else is struggling with this, try to use remapdis instead of remapbil. I tried everything in the world, but for me using remapdis fixed this

I also fixed the problem by using remapdis in CanESM5,MPI-ESM-1-2-HAM and MPI-ESM1-2-LR.

You can use remapdis but make sure you correct the direction of your interpolated velocity vectors which is still relative to the model grid, not relative to geographic coordinates.