grb_write_var_slice: File too large
Added by Haile Xue about 5 years ago
Hi there,
I would like to select a series of data (one variable) from the original 2.5 km resolution icon file by the following command
cdo -P 8 -f grb2 seltimestep,1/96/4 -selvar,sp /work/ka1081/DYAMOND/ICON-2.5km/nwp_R2B10_lkm1007_atm2_2d_ml_20160815T000000Z.grb /scratch/m/m300751/TPFDvsEASP/PS/ps0025_20160815.grb2
the input file and output file are of same format (grib2), but an error message is printed as
grb_write_var_slice: File too large
Error (grb_write_var_slice): Failed to write GRIB slice!
The output file only contains a part of the time series.
Since the output is only part of the original data, it should be smaller sized. there should be parameters to save large size grib2 files, is there?
Best,
Haile
Replies (4)
RE: grb_write_var_slice: File too large - Added by Karin Meier-Fleischer about 5 years ago
Hi Haile,
your input file is >52GB but I was able to extract the variable and timesteps with
cdo -L -f grb2 -seltimestep,1/96/4 -selvar,sp /work/ka1081/DYAMOND/ICON-2.5km/nwp_R2B10_lkm1007_atm2_2d_ml_20160815T000000Z.grb /scratch/m/m300751/TPFDvsEASP/PS/ps0025_20160815.grb2
-Karin
RE: grb_write_var_slice: File too large - Added by Haile Xue about 5 years ago
Dear Karin
the following information is just the outputted from your recommended command
m300751@mlogin105% cdo -L -f grb2 -seltimestep,1/96/4 -selvar,sp /work/ka1081/DYAMOND/ICON-2.5km/nwp_R2B10_lkm1007_atm2_2d_ml_20160815T000000Z.grb /scratch/m/m300751/TPFDvsEASP/PS/test.grb2
cdo(2) selname: Process started
error: You exceeded the file size limit for processing on mistral login nodes.
error: Please use mistralpp or batch nodes for data processing.
See
<https://www.dkrz.de/up/services/analysis/data-processing/processing-on-mistral>
for information on data processing on mistral
RE: grb_write_var_slice: File too large - Added by Karin Meier-Fleischer about 5 years ago
The error message tells you exactly what you have done wrong! You CAN NOT run the command on a mistral node you have to use mistralpp login node (that was what I did).
1. login to mistralpp
2. cdo -L -f grb2 -seltimestep,1/96/4 -selvar,sp /work/ka1081/DYAMOND/ICON-2.5km/nwp_R2B10_lkm1007_atm2_2d_ml_20160815T000000Z.grb /scratch/m/m300751/TPFDvsEASP/PS/ps0025_20160815.grb2
RE: grb_write_var_slice: File too large - Added by Haile Xue almost 5 years ago
Dear Karin
I have understood totally. Thanks