Project

General

Profile

Error merging multiple netcdf files from Era5 land data

Added by Anthony Guamán over 1 year ago

Hi,

I downloaded Era5 Land hourly data from here ( [[https://cds.climate.copernicus.eu/cdsapp#!/dataset/reanalysis-era5-land?tab=form]] ) with a temporal coverage of 2022 to near real time (February,2023) for multiple variables, so I have a data base with 14 netcdf files for each variable and I need to merge data into a single netcdf file so I can get daily averages and daily accumulated values from this data. I share with you a sample with the last 10 netcdf files.

The problem appears when I use this line of code:

cdo -b F64  mergetime 2M_TEMPERATURE_DEWPOINT_20*.nc 2M_TEMPERATURE_DEWPOINT_MERGED_HH.nc

I get the following error message:
cdo mergetime (Abort): Input streams have different number of layers per timestep!

So I tried to downloading data one more time, but the error was still there. I tested if the code works using month groups and I got that it works without problem when I only use data from Janary-2022 to November-2022, but when I use the .nc file of December-2022 I get the error message again, so I tried one more time to merge last 3 .nc files (December-2022, Janary-2023 and February-2023)as a result of this I'm getting this error again, but when I merge January-2023 and February-2023 it works. Therefore, the error is related with .nc file of december-2022 (2M_TEMPERATURE_DEWPOINT_2022_12_HH.nc).

I re-download file for december-2022 several times, but the error still appears.If anyone has any additional suggestions or thoughts on the matter, please assist.

Thanks kindly.
Anthony


Replies (6)

RE: Error merging multiple netcdf files from Era5 land data - Added by Estanislao Gavilan over 1 year ago

Hi Anthony,

I think it is because the dimensions do no match. Unlike the others, the variable d2m of the month 12 has 4 dimensions: longitude, latitude, expver and time.

Kind regards,

Estanislao

RE: Error merging multiple netcdf files from Era5 land data - Added by Anthony Guamán over 1 year ago

Thanks for your prompt response. You are right, I use "sinfo" and I can see this differences.

I downloaded all the files once again with the same specifications, but exactly at the month 12 these dimensions change.

I am new using cdo, so I do not know if there is a way to match the dimension of these files using cdo?

RE: Error merging multiple netcdf files from Era5 land data - Added by Estanislao Gavilan over 1 year ago

Hi Anthony,

I have never used land data from ERA5. If you do not know the purpose of that extra dimension, you should contact the copernicus website. They are really helpful. It could be even a bug in the dataset.

Kind regards,

Estanislao

RE: Error merging multiple netcdf files from Era5 land data - Added by Karin Meier-Fleischer over 1 year ago

Hi Anthony,

if you know which value of the added expver dimension in the 12th file you need to use you can extract the data from the file.

expver has the values 1, 5

For expver = 1:

cdo --reduce_dim -sellevel,1 2M_TEMPERATURE_DEWPOINT_2022_12_HH.nc 2M_TEMPERATURE_DEWPOINT_2022_12_HH_expver_1.nc

and for expver = 5:

cdo --reduce_dim -sellevel,5 2M_TEMPERATURE_DEWPOINT_2022_12_HH.nc 2M_TEMPERATURE_DEWPOINT_2022_12_HH_expver_5.nc

RE: Error merging multiple netcdf files from Era5 land data - Added by Estanislao Gavilan over 1 year ago

Hi,

I think this needs a bit of explanation. It seems that there is a preprocessed early ERA5 land dataset called ERA5 land T. ERA5-Land-T is data no more than three months behind real time. In contrast to ERA5, ERA5T data has flaws. Thats is why there is an extra dimension called expver. expver = 1 and expver=5 corresponds to ERA5 and to ERA5T, respectively. What I do not know if the data from 2023 corresponds to ERA5T or ERA5. It should be ERA5T because it is early data and does not have the extra dimension.

RE: Error merging multiple netcdf files from Era5 land data - Added by Anthony Guamán over 1 year ago

Hi Karin and Estanislao,

You are right!, I read the data documentation for ERA5-Land and expver=5 is the value for 2023 files that correspond to the ERA5-Land-T Near Real Time dataset (2-3 months before the present). Aditionally, the double value in the expver dimension is an error caused by considering month 12 as part of ERA5-Land (1950- 2 or 3 months before present) and ERA5-Land-T (2-3 months before present- 6 days before present), so month 12 was a clash of dates with both values in this dimension. Thank you for this clarification dear Estanislao.

Then, using Karin's recommendation to extract the expver=5 value for month, 12 I could see that the merging of the dates works perfectly, thanks a lot for your help Karin.

Thank you all so much for the prompt reply and detailed solution! It helped me a lot with this problem.

Kind regards,
Anthony

    (1-6/6)