Project

General

Profile

How to multiply constant value to variable in netcdf file

Added by nazanin tavakoli about 3 years ago

Hello,
I have a NetCDF file containing monthly cloud fraction values, and I want to multiply the variables with a constant number e.g. 100. I have tried 2 kinds of code in CDO but not only the variables do not maultiply with 100, but also the variable in each cell change.

cdo aexpr,"myvar=cc*100" cloudFraction.nc cloudiness1.nc
cdo -b F32 -mulc,100. cloudFraction.nc cloudiness.nc

Here is my main data:

thank you


Replies (20)

RE: How to multiply constant value to variable in netcdf file - Added by Ralf Mueller about 3 years ago

hi!

for me both commands work perfectly. I don't understand, what is wrong with your output. you might upload it for further analysis.

cheers
ralf

RE: How to multiply constant value to variable in netcdf file - Added by nazanin tavakoli about 3 years ago

yes, you're right. I have another question. I have two NetCDF files with different variables. One of them is Maximum Temperature, and the other one is Minimum Temperature. Firstly, I want to have an average of these two NetCDF files. So, I used the following code:
cdo ensmean TerraClimate_tmax_2001.nc TerraClimate_tmin_2001.nc sum2.nc
my error is:
cdo ensmean TerraClimate_tmax_2001.nc TerraClimate_tmin_2001.nc sum2.nc
cdo ensmean (Warning): Input streams have different parameter names!
cdf_put_att_double: ncid = 196608 varid = 5 att = _FillValue val = -8

Error (cdf_put_att_double): NetCDF: Numeric conversion not representable

Error (cdf_close): NetCDF: HDF error

Secondly, I want to minus these two variable, so I used the code below:
cdo sub TerraClimate_tmax_2001.nc TerraClimate_tmin_2001.nc sum3.nc

my error is:
cdo sub (Warning): Input streams have different parameter names!
cdf_put_att_double: ncid = 196608 varid = 5 att = _FillValue val = -8

Error (cdf_put_att_double): NetCDF: Numeric conversion not representable

Error (cdf_close): NetCDF: HDF error

Thirdly, my data is monthly through 2001. I want to divide each month into a specific constant value. For example, I want to divide the first month of my maximum temperature to 31, then the second-month divide to 28, and so on.
Could you please help me?
my data is in this link:

https://drive.google.com/file/d/1Wa3gwz9Uvo_QMS6Uy5g-aIfUz3KohjVO/view?usp=sharing

RE: How to multiply constant value to variable in netcdf file - Added by Ralf Mueller about 3 years ago

nazanin tavakoli wrote:

yes, you're right. I have another question. I have two NetCDF files with different variables. One of them is Maximum Temperature, and the other one is Minimum Temperature. Firstly, I want to have an average of these two NetCDF files. So, I used the following code:
cdo ensmean TerraClimate_tmax_2001.nc TerraClimate_tmin_2001.nc sum2.nc
my error is:
cdo ensmean TerraClimate_tmax_2001.nc TerraClimate_tmin_2001.nc sum2.nc
cdo ensmean (Warning): Input streams have different parameter names!
cdf_put_att_double: ncid = 196608 varid = 5 att = _FillValue val = -8

Error (cdf_put_att_double): NetCDF: Numeric conversion not representable

Error (cdf_close): NetCDF: HDF error

if you want a simple average of the two fields, you can compute it in the regular way mean = 0.5*(tmax+tmin)

ensmean and all the other operators for ensembles expect the ensemble members to be in different files, so this won't work for you I think.

use this instead

cdo -L -mulc,0.5 -add -delname,station_influence TerraClimate_tmax_2001.nc -delname,station_influence TerraClimate_tmin_2001.nc mean.nc

Secondly, I want to minus these two variable, so I used the code below:
cdo sub TerraClimate_tmax_2001.nc TerraClimate_tmin_2001.nc sum3.nc

my error is:
cdo sub (Warning): Input streams have different parameter names!
cdf_put_att_double: ncid = 196608 varid = 5 att = _FillValue val = -8

Error (cdf_put_att_double): NetCDF: Numeric conversion not representable

Error (cdf_close): NetCDF: HDF error

Use

cdo -L  -sub -delname,station_influence TerraClimate_tmax_2001.nc -delname,station_influence TerraClimate_tmin_2001.nc sub.nc

the station data is unsigned integer which should not be substracted. so you have to delete this variable from the stream befire substracting

Thirdly, my data is monthly through 2001. I want to divide each month into a specific constant value. For example, I want to divide the first month of my maximum temperature to 31, then the second-month divide to 28, and so on.
Could you please help me?
my data is in this link:

https://drive.google.com/file/d/1Wa3gwz9Uvo_QMS6Uy5g-aIfUz3KohjVO/view?usp=sharing

for the third request I need to think a bit ...

RE: How to multiply constant value to variable in netcdf file - Added by Ralf Mueller about 3 years ago

cdo -infov -monsum -settaxis,2001-01-01,12:00:00,1days -expr,'seq=seq/seq' -seq,1,365

This call illustrated how to get these monthly-weights or length-of-month: you simple use a constant field with value 1.0 and sum it up for each month (monsum). seq created a time series with increasing values, but the usage of 'seq=seq/seq' sets it to 1.0. then you need to specify the timeaxis so that it fit with your target data set (here: monthly values)

But this is just a one-point-grid. For putting it on the target grid, you need to use the enlarge operator. the final call looks like:

cdo -L -div -selname,tmax TerraClimate_tmax_2001.nc -enlarge,TerraClimate_tmax_2001.nc -monsum -settaxis,2001-01-01,12:00:00,1days -expr,'seq=seq/seq' -seq,1,365 monthMul.nc  

hth
ralf

RE: How to multiply constant value to variable in netcdf file - Added by nazanin tavakoli about 3 years ago

I have this error:

Operator >seq< not found!
Similar operators are:
eq eqc sqr

RE: How to multiply constant value to variable in netcdf file - Added by Ralf Mueller about 3 years ago

then you use an older CDO version. you can use the for operator instead

RE: How to multiply constant value to variable in netcdf file - Added by nazanin tavakoli about 3 years ago

I am using version 1.9.4. could you please tell me how to update it in windows 10?

RE: How to multiply constant value to variable in netcdf file - Added by Ralf Mueller about 3 years ago

do you use the Linux subsystem or cygwin?

RE: How to multiply constant value to variable in netcdf file - Added by nazanin tavakoli about 3 years ago

I have installed cygwin64 but I am coding in windows batch file

RE: How to multiply constant value to variable in netcdf file - Added by nazanin tavakoli about 3 years ago

Thank you for your help.
I have another question about netcdf4 files. I have 30 netcdf4 files which include daily precipitation data, and I want to calculate "wet" days per month data. The WET variable represents counts of wet days defined as having ≥0.1 mm of Precipitation for each cell. Could you please tell me how can I do it in CDO?
I have sent 2 of my data in the following link, and the main variable is pericipitationCal.

https://drive.google.com/file/d/1Ue61GyXjLF2ktUHIpp4t4VjjIgQjAN1F/view?usp=sharing

Thank you

RE: How to multiply constant value to variable in netcdf file - Added by Ralf Mueller about 3 years ago

check the climate indices coming with CDO "here":projects/cdo/embedded/cdo_eca.pdf

you could do the calculation on your own by creating a 0-1-mask based on the 0.1mm threshold (operators ltc or gtc) and sum up this mask over each month

hth
ralf

RE: How to multiply constant value to variable in netcdf file - Added by nazanin tavakoli about 3 years ago

how can I create a mask and sum up them for each month? Could you please help me?

RE: How to multiply constant value to variable in netcdf file - Added by nazanin tavakoli about 3 years ago

I should use this code for calculating wet days per month which is the count of days having >=0.1 mm prcp?

cdo eca_pd,0.1 infile outfile

The infile should be a monthly data? I have daily data, so I should merge them into a single file as an input of the code?

RE: How to multiply constant value to variable in netcdf file - Added by Ralf Mueller about 3 years ago

the eca_* operators expect daily input data. please check the manual with

cdo -h eca_pd

please check the operators ltc and gtc for creating masks out of input data (https://code.mpimet.mpg.de/projects/cdo/embedded/index.html#x1-2170002.5.2)

RE: How to multiply constant value to variable in netcdf file - Added by nazanin tavakoli about 3 years ago

I used this code ( cdo eca_pd,0.1 3B-DAY.MS.MRG.3IMERG.20010101-S000000-E235959.V06.nc4 day1.nc) for one day, but it doesn't count cells which have 0.1 mm prcp. Also, I used this code (cdo gec,0.1 3B-DAY.MS.MRG.3IMERG.20010101-S000000-E235959.V06.nc4 day2.nc), but the output file doesn't consider the cells which have 0.1 mm prcp. Do you have any idea how can I fix this?

RE: How to multiply constant value to variable in netcdf file - Added by Ralf Mueller about 3 years ago

The operator does not sum up locations for which the precipitation is above a threshold but time steps. I have to admit that I still don't fully understand what you want to achieve

  • do you want to sum up time steps?
  • do you want to sum up horizontal locations?

https://www.ecad.eu/indicesextremes/indicesdictionary.php official definitions of eca

RE: How to multiply constant value to variable in netcdf file - Added by nazanin tavakoli about 3 years ago

I do not want to sum up locations. I want to calculate numbers of day which have more and equal than 0.1 mm precipitation( >=0.1mm) like the RR1 index which has explained in the link that you have sent for me in the last massage. I have 30 netcdf4 daily precipitation data, and in the end, I want a NetCDF file that contains counts of wet days defined as having ≥0.1 mm of Precipitation in each cell. I have 2 problems when I used the following codes( I did not use them at the same time, I mean that these are 2 completely different codes).

cdo eca_pd,0.1 3B-DAY.MS.MRG.3IMERG.20010101-S000000-E235959.V06.nc4 day1.nc

cdo gec,0.1 3B-DAY.MS.MRG.3IMERG.20010101-S000000-E235959.V06.nc4 day2.nc

First of all, they did not count when a cell has 0.1 mm precipitation. I mean that it seems that they just count when the cell has more than 0.1 mm precipitation.
Secondly, my main netcdf4 data contains some parameters, and I want to calculate wet days just for the precipitationCal variable.

I have sent my sample data in the last massage.
I would appreciate it if you help me.

RE: How to multiply constant value to variable in netcdf file - Added by Ralf Mueller about 3 years ago

Referring to your zipfile from google drive, the input looks like this to me:

  • first I merge the two files together:
    cdo  -cat 3B-DAY.MS.MRG.3IMERG.20010101-S000000-E235959.V06.nc4 3B-DAY.MS.MRG.3IMERG.20010102-S000000-E235959.V06.nc4 in.nc                                                                                                                                                                                               
    Warning (cdfScanVarAttr): NetCDF: Variable not found - >nv<
    Warning (cdf_set_var): Inconsistent variable definition for time_bnds!
    Warning (cdf_set_calendar): calendar >julian< unsupported!
    cdo    cat:  50%Warning (cdfScanVarAttr): NetCDF: Variable not found - >nv<
    Warning (cdf_set_var): Inconsistent variable definition for time_bnds!
    Warning (cdf_set_calendar): calendar >julian< unsupported!
    cdo    cat: Processed 103680000 values from 16 variables over 2 timesteps [4.00s 455MB].
  • now lets look into it
    ❯ cdo infov in.nc
        -1 :       Date     Time   Level Gridsize    Miss :     Minimum        Mean     Maximum : Parameter name
         1 : 2001-01-01 00:00:00       0  6480000 1770999 :      0.0000      3.1743      638.91 : precipitationCal
         2 : 2001-01-01 00:00:00       0  6480000       0 :      0.0000      34.879      48.000 : precipitationCal_cnt
         3 : 2001-01-01 00:00:00       0  6480000       0 :      0.0000      6.0597      48.000 : precipitationCal_cnt_cond
         4 : 2001-01-01 00:00:00       0  6480000   43414 :      0.0000     0.37036      171.14 : HQprecipitation
         5 : 2001-01-01 00:00:00       0  6480000       0 :      0.0000      10.406      26.000 : HQprecipitation_cnt
         6 : 2001-01-01 00:00:00       0  6480000       0 :      0.0000     0.71670      19.000 : HQprecipitation_cnt_cond
         7 : 2001-01-01 00:00:00       0  6480000 1770999 :     0.11906      4.6546      271.98 : randomError   
         8 : 2001-01-01 00:00:00       0  6480000       0 :      0.0000      34.879      48.000 : randomError_cnt
         9 : 2001-01-02 00:00:00       0  6480000 1768762 :      0.0000      3.0071      1306.7 : precipitationCal
        10 : 2001-01-02 00:00:00       0  6480000       0 :      0.0000      34.896      48.000 : precipitationCal_cnt
        11 : 2001-01-02 00:00:00       0  6480000       0 :      0.0000      5.8177      48.000 : precipitationCal_cnt_cond
        12 : 2001-01-02 00:00:00       0  6480000   43372 :      0.0000     0.37901      275.07 : HQprecipitation
        13 : 2001-01-02 00:00:00       0  6480000       0 :      0.0000      10.693      27.000 : HQprecipitation_cnt
        14 : 2001-01-02 00:00:00       0  6480000       0 :      0.0000     0.73538      22.000 : HQprecipitation_cnt_cond
        15 : 2001-01-02 00:00:00       0  6480000 1768762 :     0.11906      4.4880      406.00 : randomError   
        16 : 2001-01-02 00:00:00       0  6480000       0 :      0.0000      34.896      48.000 : randomError_cnt
    cdo    infon: Processed 103680000 values from 8 variables over 2 timesteps [1.36s 305MB].
    

So you have multiple precipitation variables ranging from 0 to 640mm. Now let's try the eca_pd operator:

 cdo infov -eca_pd,0.1 in.nc
cdo(1) eca_pd: Process started
    -1 :       Date     Time   Level Gridsize    Miss :     Minimum        Mean     Maximum : Parameter name
     1 : 2001-07-02 12:00:00       0  6480000 1762303 :      0.0000     0.93970      2.0000 : precipitation_days_index_per_time_period
cdo(1) eca_pd: Processed 12960000 values from 8 variables over 2 timesteps.
cdo    infon: Processed 6480000 values from 1 variable over 1 timestep [0.68s 384MB].

since you have values way above 0.1mm and only two days of data, you get a max value of 2. This looks totally fine to me. eca_pd only takes into account the first variable in the input (precipitationCal)

Next let's emulate the calculation of the number of days with precip with the gec operator followed by the summation over all timesteps:

cdo infov -timsum -gec,0.1 in.nc
cdo(1) timsum: Process started
cdo(2) gtc: Process started
    -1 :       Date     Time   Level Gridsize    Miss :     Minimum        Mean     Maximum : Parameter name
     1 : 2001-01-01 12:00:00       0  6480000 1762303 :      0.0000     0.93970      2.0000 : precipitationCal
     2 : 2001-01-01 12:00:00       0  6480000       0 :      0.0000      1.4537      2.0000 : precipitationCal_cnt
     3 : 2001-01-01 12:00:00       0  6480000       0 :      0.0000     0.89767      2.0000 : precipitationCal_cnt_cond
     4 : 2001-01-01 12:00:00       0  6480000   42729 :      0.0000     0.36552      2.0000 : HQprecipitation
     5 : 2001-01-01 12:00:00       0  6480000       0 :      0.0000      1.9866      2.0000 : HQprecipitation_cnt
     6 : 2001-01-01 12:00:00       0  6480000       0 :      0.0000     0.51300      2.0000 : HQprecipitation_cnt_cond
     7 : 2001-01-01 12:00:00       0  6480000 1762303 :      1.0000      1.9968      2.0000 : randomError   
     8 : 2001-01-01 12:00:00       0  6480000       0 :      0.0000      1.4537      2.0000 : randomError_cnt
cdo(2) gtc: Processed 103680000 values from 8 variables over 2 timesteps.
cdo(1) timsum: Processed 103680000 values from 8 variables over 2 timesteps.
cdo    infon: Processed 51840000 values from 8 variables over 1 timestep [2.48s 936MB].
Here you have numbers for all input variables. If that makes sense for your variables is up to you, but obviously the values for precipitationCal is identical to the usage of eca_pd

gec only creates a 0-1-mask, so in order to get the number of days, you have to sum up this mask.

hth
ralf

    (1-20/20)