multiplying shapefile in netcdf format with netcdf files that has the data gives error.
Added by sara Abdelaziz over 3 years ago
Hi, can you help me in identifying the problem I have when trying to multiply the region.nc which is a shapefile converted into a .nc file to mask the wsgmax10m_rcp85_land-cpm_uk_2.2km_01_1hr_19801201-19801230.nc file? I use cdo -mul to do the mask
the wsgmac...nc file: https://drive.google.com/file/d/1Voj6CzmPgWvFs5OKjbHwGVdmDZm1NEPJ/view?usp=sharing
the converted shapefile to .nc file: https://drive.google.com/file/d/1f5MI6J58yaavKJ7LRy9tRyEIrevUtX-J/view?usp=sharing
the error I am having is:
Warning (cdfScanVarAttr): Time must be the first dimension! Unsupported array structure, skipped variable wsgmax10m! Warning (cdf_set_var): Inconsistent variable definition for wsgmax10m!
Warning (cdfCheckVars): Unsupported data type (char/string), skipped variable yyyymmddhh!
Warning (cdfInqContents): Coordinates variable ensemble_member_id can't be assigned!
Warning (cdfInqContents): Coordinates variable month_number can't be assigned! cdo mul (Abort): Grid size of the input parameters do not match!
Replies (8)
RE: multiplying shapefile in netcdf format with netcdf files that has the data gives error. - Added by Karin Meier-Fleischer over 3 years ago
Hi Sara,
the problem is that your data file is not CF conform. The variable wsgmax10m has dimensions (ensemble_member,time,lat,lon) and also the coordinates attribute is wrong. The shapefile variable Band1 is stored on another grid than wsgmax10m. Therefore, you have to do some work before you can multiply the data and the shapefile content.
1. Use NCO's ncwa routine to 'delete' the dimension ensemble_member:
ncwa -O -a ensemble_member wsgmax10m_rcp85_land-cpm_uk_2.2km_01_1hr_19801201-19801230.nc tmp1.nc
2. Use NCO's ncatted to set the correct coordinates attribute of wsgmax10m variable:
ncatted -O -a coordinates,wsgmax10m,o,c,"latitude longitude" tmp1.nc tmp2.nc
3. Use CDO to remap shapefile Band1 to the same grid as wsgmax10m and then do the multiplication:
cdo -mul -selname,wsgmax10m tmp2.nc -remapbil,tmp2.nc region.nc outfile.nc
-Karin
RE: multiplying shapefile in netcdf format with netcdf files that has the data gives error. - Added by sara Abdelaziz over 3 years ago
Thank you so much for your answer, I have been working on it to find the solution and I came up with this solution:
1- I remaped the shapefile
cdo remapbil,wsgmax10m_rcp85_land-cpm_uk_2.2km_01_1hr_19801201-19801230.nc region.nc remapbil_region.nc
2-mean over ensembel_member to delete it
ncwa a ensemble_member wsgmax10m_rcp85_land-cpm_uk_2.2km_01_1hr_19801201-19801230.nc tmp1.nc delete all the unnecessarly variabels
3
ncks C -O -x -v ensemble_member,month_number,year,yyyymmddhh,ensemble_member_id tmp1.nc tmp2.nc do the multiplication
4
cdo -mul tmp22.nc remapbil_region.nc outfile2.nc
I think your solution is much efficient as it's not required to delete and variables from the original sgmax10m_rcp85.nc file.
RE: multiplying shapefile in netcdf format with netcdf files that has the data gives error. - Added by sara Abdelaziz about 3 years ago
Karin Meier-Fleischer wrote in RE: multiplying shapefile in netcdf format with netcdf f...:
Hi Sara,
the problem is that your data file is not CF conform. The variable wsgmax10m has dimensions (ensemble_member,time,lat,lon) and also the coordinates attribute is wrong. The shapefile variable Band1 is stored on another grid than wsgmax10m. Therefore, you have to do some work before you can multiply the data and the shapefile content.
1. Use NCO's ncwa routine to 'delete' the dimension ensemble_member:
[...]
2. Use NCO's ncatted to set the correct coordinates attribute of wsgmax10m variable:
[...]
3. Use CDO to remap shapefile Band1 to the same grid as wsgmax10m and then do the multiplication:
[...]
-Karin
Hi, May I have your help with the resulted netcdf file from the above task, please? as I wanted to convert the rotated curvilinear grid into a regular lat lon grid with the same 2.2km resolution in the original file.
(I attached the new masked grid to the question):
https://drive.google.com/file/d/1T3HsBpWc_J8Y5cVxxZ_GagDoa3GFxjYC/view?usp=sharing
RE: multiplying shapefile in netcdf format with netcdf files that has the data gives error. - Added by Karin Meier-Fleischer about 3 years ago
Hi Sara,
which one is the original file that you want to use for regridding? I'm a little bit confused.
file to be regridded = drive.google.com/file/d/1T3HsBpWc_J8Y5cVxxZ_GagDoa3GFxjYC/view?usp=sharing
use lonlat grid of file = ?
Usually the following is done
cdo -remapbil,your-lonlat-grid-file infile outfile
-Karin
RE: multiplying shapefile in netcdf format with netcdf files that has the data gives error. - Added by sara Abdelaziz about 3 years ago
Hi Karin,
I have the file to be gridded, which is (sfcWind_sfcWind_rcp85_land-cpm_uk_2.2km_01_day_19991201-20001130_mask) but I am not sure how to create the latlon grid file with 2.2km resolution similar to the file that I want to regrid to be used to convert curvilinear to a regular grid. I saw an answer using
cdo setgridtype) but I am not sure how to function it in my case.
Thank you.
RE: multiplying shapefile in netcdf format with netcdf files that has the data gives error. - Added by sara Abdelaziz about 3 years ago
Hi Karin,
I tried using the below line to convert curvilinear to regular grid while maintaining the same resolution of the original file which was 2.2km, the lat: 607 and lon:484 (the link to the outfile3 is here: https://drive.google.com/file/d/16o-R3OFBzcCdee2rq1nAFHREOZC8pi-v/view?usp=sharing)
cdo -remapbil,r484x607 sfcWind_sfcWind_rcp85_land-cpm_uk_2.2km_01_day_20791201-20801130_mask.nc outfile3.nc
I can see some differences in the map and the scale, is this the right way to do it?
Thank you
RE: multiplying shapefile in netcdf format with netcdf files that has the data gives error. - Added by Karin Meier-Fleischer about 3 years ago
Hi Sara,
using -remapbil,r484x607 returns a global grid of 484x607 grid cells. But your data file contains a small region with 484x607 grid cells.
For remapping a region you should use a grid description file instead.
gridfile.txt
gridtype = lonlat gridsize = 293788 xsize = 484 ysize = 607 xname = longitude xlongname = "longitude" xunits = "degrees_east" yname = latitude ylongname = "latitude" yunits = "degrees_north" xfirst = -14.60127 xinc = 0.042931 yfirst = 47.42323 yinc = 0.021712
Remap the data file using gridfile.txt
cdo -remapnn,gridfile.txt sfcWind_sfcWind_rcp85_land-cpm_uk_2.2km_01_day_19991201-20001130_mask.nc outfile_sfcWind.nc
RE: multiplying shapefile in netcdf format with netcdf files that has the data gives error. - Added by sara Abdelaziz about 3 years ago
Thank you so much for your help. I tried it and it works.
Also, I tried to take the output from regrinding to the regular grid that you provided and apply another step, which is to remap it from the 2.2km (sfc_mask_remapnnregrid_era5_gridfile.nc) to another grid which is approximately 27.5 km grid (wind1_day_mask_remapcon.nc) using three different mapping methods (remapcon, remapbil, remapnn) but none of them gave a satisfying answer in term of the accuracy of edges of the plot, maybe the remapcon was the best of them.
Is there is another way to regrid with higher accuracy?
cdo -remapcon,wind2_day_mask_remapcon.nc sfc_mask_remapnnregrid_era5_gridfile.nc sfc_gridfile__remapconregrid.nc
sfc_mask_remapnnregrid_era5_gridfile.nc: https://drive.google.com/file/d/1mL7ku05ErvCika8gjNwHJjDHKYTIddBG/view?usp=sharing
wind2_day_mask_remapcon.nc: https://drive.google.com/file/d/1CuA2vvPu710FzWBhOlHSx66ieMK3u43D/view?usp=sharing
sfc_gridfile__remapconregrid.nc: https://drive.google.com/file/d/1SdsHVJaQ6mvaJ8pebAvzN5Lr_35olom4/view?usp=sharing
Thank you in advance, you have been a great support.
Sara