Project

General

Profile

multiplying shapefile in netcdf format with netcdf files that has the data gives error.

Added by sara Abdelaziz over 2 years ago

Hi, can you help me in identifying the problem I have when trying to multiply the region.nc which is a shapefile converted into a .nc file to mask the wsgmax10m_rcp85_land-cpm_uk_2.2km_01_1hr_19801201-19801230.nc file? I use cdo -mul to do the mask

the wsgmac...nc file: https://drive.google.com/file/d/1Voj6CzmPgWvFs5OKjbHwGVdmDZm1NEPJ/view?usp=sharing
the converted shapefile to .nc file: https://drive.google.com/file/d/1f5MI6J58yaavKJ7LRy9tRyEIrevUtX-J/view?usp=sharing

the error I am having is:

Warning (cdfScanVarAttr): Time must be the first dimension! Unsupported array structure, skipped variable wsgmax10m! Warning (cdf_set_var): Inconsistent variable definition for wsgmax10m!
Warning (cdfCheckVars): Unsupported data type (char/string), skipped variable yyyymmddhh!
Warning (cdfInqContents): Coordinates variable ensemble_member_id can't be assigned!
Warning (cdfInqContents): Coordinates variable month_number can't be assigned! cdo mul (Abort): Grid size of the input parameters do not match!


Replies (8)

RE: multiplying shapefile in netcdf format with netcdf files that has the data gives error. - Added by Karin Meier-Fleischer over 2 years ago

Hi Sara,

the problem is that your data file is not CF conform. The variable wsgmax10m has dimensions (ensemble_member,time,lat,lon) and also the coordinates attribute is wrong. The shapefile variable Band1 is stored on another grid than wsgmax10m. Therefore, you have to do some work before you can multiply the data and the shapefile content.

1. Use NCO's ncwa routine to 'delete' the dimension ensemble_member:

ncwa -O -a ensemble_member wsgmax10m_rcp85_land-cpm_uk_2.2km_01_1hr_19801201-19801230.nc tmp1.nc

2. Use NCO's ncatted to set the correct coordinates attribute of wsgmax10m variable:

ncatted -O -a coordinates,wsgmax10m,o,c,"latitude longitude" tmp1.nc tmp2.nc

3. Use CDO to remap shapefile Band1 to the same grid as wsgmax10m and then do the multiplication:

cdo -mul -selname,wsgmax10m tmp2.nc -remapbil,tmp2.nc region.nc outfile.nc

-Karin

RE: multiplying shapefile in netcdf format with netcdf files that has the data gives error. - Added by sara Abdelaziz over 2 years ago

Thank you so much for your answer, I have been working on it to find the solution and I came up with this solution:
1- I remaped the shapefile
cdo remapbil,wsgmax10m_rcp85_land-cpm_uk_2.2km_01_1hr_19801201-19801230.nc region.nc remapbil_region.nc
2-mean over ensembel_member to delete it
ncwa a ensemble_member wsgmax10m_rcp85_land-cpm_uk_2.2km_01_1hr_19801201-19801230.nc tmp1.nc
3
delete all the unnecessarly variabels
ncks C -O -x -v ensemble_member,month_number,year,yyyymmddhh,ensemble_member_id tmp1.nc tmp2.nc
4
do the multiplication
cdo -mul tmp22.nc remapbil_region.nc outfile2.nc

I think your solution is much efficient as it's not required to delete and variables from the original sgmax10m_rcp85.nc file.

RE: multiplying shapefile in netcdf format with netcdf files that has the data gives error. - Added by sara Abdelaziz about 2 years ago

Karin Meier-Fleischer wrote in RE: multiplying shapefile in netcdf format with netcdf f...:

Hi Sara,

the problem is that your data file is not CF conform. The variable wsgmax10m has dimensions (ensemble_member,time,lat,lon) and also the coordinates attribute is wrong. The shapefile variable Band1 is stored on another grid than wsgmax10m. Therefore, you have to do some work before you can multiply the data and the shapefile content.

1. Use NCO's ncwa routine to 'delete' the dimension ensemble_member:

[...]

2. Use NCO's ncatted to set the correct coordinates attribute of wsgmax10m variable:

[...]

3. Use CDO to remap shapefile Band1 to the same grid as wsgmax10m and then do the multiplication:

[...]

-Karin

Hi, May I have your help with the resulted netcdf file from the above task, please? as I wanted to convert the rotated curvilinear grid into a regular lat lon grid with the same 2.2km resolution in the original file.
(I attached the new masked grid to the question):

https://drive.google.com/file/d/1T3HsBpWc_J8Y5cVxxZ_GagDoa3GFxjYC/view?usp=sharing

RE: multiplying shapefile in netcdf format with netcdf files that has the data gives error. - Added by Karin Meier-Fleischer about 2 years ago

Hi Sara,

which one is the original file that you want to use for regridding? I'm a little bit confused.

file to be regridded = drive.google.com/file/d/1T3HsBpWc_J8Y5cVxxZ_GagDoa3GFxjYC/view?usp=sharing

use lonlat grid of file = ?

Usually the following is done

cdo -remapbil,your-lonlat-grid-file infile outfile

-Karin

RE: multiplying shapefile in netcdf format with netcdf files that has the data gives error. - Added by sara Abdelaziz about 2 years ago

Hi Karin,
I have the file to be gridded, which is (sfcWind_sfcWind_rcp85_land-cpm_uk_2.2km_01_day_19991201-20001130_mask) but I am not sure how to create the latlon grid file with 2.2km resolution similar to the file that I want to regrid to be used to convert curvilinear to a regular grid. I saw an answer using
cdo setgridtype) but I am not sure how to function it in my case.
Thank you.

RE: multiplying shapefile in netcdf format with netcdf files that has the data gives error. - Added by sara Abdelaziz about 2 years ago

Hi Karin,

I tried using the below line to convert curvilinear to regular grid while maintaining the same resolution of the original file which was 2.2km, the lat: 607 and lon:484 (the link to the outfile3 is here: https://drive.google.com/file/d/16o-R3OFBzcCdee2rq1nAFHREOZC8pi-v/view?usp=sharing)
cdo -remapbil,r484x607 sfcWind_sfcWind_rcp85_land-cpm_uk_2.2km_01_day_20791201-20801130_mask.nc outfile3.nc
I can see some differences in the map and the scale, is this the right way to do it?

Thank you

RE: multiplying shapefile in netcdf format with netcdf files that has the data gives error. - Added by Karin Meier-Fleischer about 2 years ago

Hi Sara,

using -remapbil,r484x607 returns a global grid of 484x607 grid cells. But your data file contains a small region with 484x607 grid cells.

For remapping a region you should use a grid description file instead.

gridfile.txt

gridtype  = lonlat
gridsize  = 293788
xsize     = 484
ysize     = 607
xname     = longitude
xlongname = "longitude" 
xunits    = "degrees_east" 
yname     = latitude
ylongname = "latitude" 
yunits    = "degrees_north" 
xfirst    = -14.60127
xinc      = 0.042931
yfirst    = 47.42323
yinc      = 0.021712

Remap the data file using gridfile.txt

cdo -remapnn,gridfile.txt sfcWind_sfcWind_rcp85_land-cpm_uk_2.2km_01_day_19991201-20001130_mask.nc outfile_sfcWind.nc

RE: multiplying shapefile in netcdf format with netcdf files that has the data gives error. - Added by sara Abdelaziz about 2 years ago

Thank you so much for your help. I tried it and it works.
Also, I tried to take the output from regrinding to the regular grid that you provided and apply another step, which is to remap it from the 2.2km (sfc_mask_remapnnregrid_era5_gridfile.nc) to another grid which is approximately 27.5 km grid (wind1_day_mask_remapcon.nc) using three different mapping methods (remapcon, remapbil, remapnn) but none of them gave a satisfying answer in term of the accuracy of edges of the plot, maybe the remapcon was the best of them.
Is there is another way to regrid with higher accuracy?

cdo -remapcon,wind2_day_mask_remapcon.nc sfc_mask_remapnnregrid_era5_gridfile.nc sfc_gridfile__remapconregrid.nc

sfc_mask_remapnnregrid_era5_gridfile.nc: https://drive.google.com/file/d/1mL7ku05ErvCika8gjNwHJjDHKYTIddBG/view?usp=sharing
wind2_day_mask_remapcon.nc: https://drive.google.com/file/d/1CuA2vvPu710FzWBhOlHSx66ieMK3u43D/view?usp=sharing
sfc_gridfile__remapconregrid.nc: https://drive.google.com/file/d/1SdsHVJaQ6mvaJ8pebAvzN5Lr_35olom4/view?usp=sharing

Thank you in advance, you have been a great support.
Sara

    (1-8/8)