CDO

What about the import operator?

Thank you, Ralf
My problem is that the data are not in regular lat lon grid. In the case of regular grid I could use for example the command:
cdo -f grb input,r32x16 ofile.grb < my_ascii_data
But what happens when I have for example longitudes: 1.5, 2.5, 3, 3.2 and latitudes 42.2, 44.4, 42.3, 44.0
Thanks

For data on irregular grids you have to create a grid description file for the grid points. Here is an example:

cdo -f nc input,my_gridfile ofile.nc < my_ascii_data

225 300 250 ...

gridtype = cell
gridsize = 4
xvals    = 1.5 2.5 3 3.2
yvals    = 42.2 44.4 42.3 44.0

Note that you can't store data on a irregular grid to GRIB.

Regards, Uwe

Ok, Uwe, this is the answer which can solve the problem!
Thank you very much!!

Uwe,
What is the format of the 'my_ascii_data' file to be the imput in the cdo command?
I mean the 225 300 250 ... data.
I guess it should be a matrix. But, What should be the format of this matrix? A diagonal one?
Thanks

I think I have solved the problem and would need a confirmation from you that the solution is correct:
I have this example file:

lon lat value
20 75 2000
12 55 1000
50 65 5000
40 85 2000
30 35 3000

Now I create this 'gridlonlat' file:

gridtype = lonlat
xsize = 5
ysize = 5
xvals = 20 12 50 40 30
yvals = 75 55 65 85 35

And create the diagonal matrix file 'data.txt':

2000 0 0 0 0
0 1000 0 0 0
0 0 5000 0 0
0 0 0 2000 0
0 0 0 0 3000

Finally: cdo -f nc input,gridlonlat ofile.nc < data.txt

It seems for me that the ofile.nc is what I need representing data from an irregular grid.
Do you think this solution is correct? (As you can see I have used gridtype=lonlat intead of cell)
Thank you very much!

Your solution is correct if you really need it with gridtype lonlat. But it is not recommended to use it in this way because normally the xvals/yvals have to increase monotonously.
If you use gridtype 'cell' then the ASCII file is unformatted and the values can have any order:

2000 1000 5000 2000 3000

2000 1000 5000
2000
3000

2000
1000
5000
2000
3000

Thank you, Uwe. I don't really need gridtype lonlat. This solution is much better!

And how do I visualize the resulting ofile.nc if using gridtype = cell?

ncview disregards longitude and makes a 2D plot with latitude and value.

I want a map...

The diagonal approach with gridtype=latlon works but I have 100 000 points, so it does not make much sense!!

Do I have to make some changes in my_gridfile or use another visualizer?

Which one?

ncl does a fairly good job for this. Look out for sfXArray and sfYArray for setting coordinate values. The examples for visualizing ICON data might be helpful, ICON used a global triangular grid.

Ralf,
Thank you very much for your response. I'm trying to get ncl, but after a long process of different registrations and subscriptions I was able to get a download script called wget-download.sh which does not work because it needs a MyProxyLogon java that uses port 7512 which is blocked by my enterprise firewall.
Of course I could do this at home, but I'm afraid if (just for doing a simple download) the process is so painfull, then the use of the software must be anything but user friendly.
I'm sure I will look at it in the near future because it seems very powerfull, but for now I just wanted to create a netcdf database with several netcdf files that could be easily acessed by a simple visualizer that could display a map just like hdfview does...
Can you help a little bit more?

Update:
For each central latlon point, I have information about dlat and dlon, so I know exactly the size of the rectangle that must be painted in the map...
After all, I must think in this more carefully, because a simple visualizer won't plot this as it should...
So, I must forget the use of a simple visualizer...
:-(
ncl paints the rectangles using dlat and dlon information in each latlon central point?

AFAIK ncl has it's own triangulation algor. But you can use the bounds of the coordinates of you cell to paint the grid. Do you want a contour plot or a grid plot?

since there are so many points a contour plot or a grid plot would be the same... Which one is easier... Any ideas?

For ncl, the contour plot is definitely easier, because there is not built-in procedure to plot the bounds of an arbitrary grid. You should start from the first ICON example to get something running.

it's running now...
I can see a interpolated contour map and I didn't have to use the dlat and dlon information
I asked for help to a colleague here that uses ncl...
Thanks a lot
But this is not the most universal way to create a map of a irregular netcdf file.
If I want to send a netcdf file to someone, I have to send also the ncl script and ask for him to install ncl which is not practical...
One idea was to remap the irregular netcdf to a regular latlon netcdf using netcd like this:

cdo remapbil,regular_grid irregular.nc regular.nc

with regular_grid like this:

gridtype=lonlat
xsize=100
ysize=100
xfirst=-39.9
xinc=0.05
yfirst=35.1
yinc=0.05

But this aborts with the error:

Internal problem, grid1 rank = 1!

However, It writes some bytes to the output file regular.nc

update:

remapnn works!!

it works but remapnn creates values for the whole domain.

In my case there are huge areas without data and I don't want to extrapolate based in values that are too far!

The idea was getting the nearest neighbour but only if the algorithm finds some value in, say, 0.05 degress radius.

It it does not find any, then fill the lalon point with nodata

is this possible?

The areas without data should contain missing values (netcdf: _FillValue) and not a special value, which might be out of your regular data range. There is group of operators which can handle these settings. If this is correctly set up, the missing data area will be taken into account while computing the interpolation weights. My recommendation is to use remapnn to a regular but finer resolution - this way, the original values are untouched and you might see the original grid. For a quick look, ncview does a good job, esp. the animation feature.

Let me explain better (for a Sea Surface Temperature map):
The irregular netcdf have a fine resolution (0.05º) over the whole domain if there are no clouds and if the satellite at that hour covers the whole domain.
If there are clouds over a vast region (which varies from day to day) then the irregular ascii/netcdf only have partial values over regions without clouds.
I don't know in advance which are the regions with clouds and such information IS NOT originally given with ANY MISSING/SPECIAL value. Simply, it does not appear in the file...
I can spot the regions with clouds, because those pixels are not painted with a color in ncl.
So I think the only way to visualize this properly in ncview is to interpolate this to latlon giving missing value if the interpolated latlon point is far from the original nearest point.
In the extreme, if I consider 0.05º already as far, then I will have lots of missing values in the file but I will be able to visualize it properly in ncview as I would with ncl using a grid plot.
The problem is how to define how far in remapnn...it should be possible!!
I'm sending the original irregular netcdf...

Is this making sense to you? If not, please tell me as soon as possible and I will find another solution to remap other then cdo.
It's a pitty because in almost all interpolating software using nearest neighbour I can choose a radius of influence.
Unfortunately not all of them reads netcdf...

Hi,

I think your "regular_grid" grid description file is inconsistent with what you really want (e.g. no of grid points, if the resolution is 0.05). Try following:

cdo -f nc -sellonlatbox,-40,5,35,45 -random,r7200x3600 lonlat.nc

will create a regular latlon grid "template" of 0.05 degree with approximate bounds as in your input file.

cdo -f nc remapnn,lonlat.nc -setmissval,-9e33 ofile.nc test.nc

will remap correctly (may take some time). You can visuallize it with ncview or Panoply - looks fine to me

Cheers,
J

Hi Jaison,
Thanks for you answer.
I've tried what you said, but it doesn't work do way I want...
My hope was this -setmissval,-9e33
My you don't have any!

In fact the regular_grid I've mentioned above will interpolate only a part of my original domain, but this smaller area is enough to see that I don't want such interpolation, because it is getting points too far!!
The right "regular_grid" for the whole domain is:

gridtype=lonlat
xsize=901
ysize=201
xfirst=-40
xinc=0.05
yfirst=35
yinc=0.05

With this and just doing:

cdo remapnn,regular_grid ofile.nc test.nc

I get exactly the same test.nc as you and I can see it in ncview.

BUT look at the pattern!! Does this look like a nice SST pattern to you?!
I don't want it to extrapolate Sea Surface Temperature over the whole domain! I just want to interpolate points where there are observations nearby!
Look at the right side of your map! You can see values in there that have a completely creepy pattern! The original irregular ofile.nc does not have information in there! Guess why?
Yes... You are right! Because its land!!

A land-sea mask won't work, because the problem is not only over land. As I already explained, it is also because of the clouds which are not always at the same place.

Hi,

attached is a regular netcdf created from your data - final attempt Please have a look at it with ncview/... and let me if this is what you are looking for.

Cheers,
J