How to solve a white spot at the 1980-09-12 day in a NetCDF4 dataset which prevents me to operate anything
Added by José Carlos Barbosa Neto 8 months ago
Hello everyone.
After detrend a +13000 dataset, I cannot operate anything due to a bug at the 1980-09-12 day. It's only this one. In the attached photo you'll see the same day: at the right it's a print I took using ncview tools, at the left it's the SAME photo using the ncview too but with the normal viewer.
As a big dataset, I could not use
cdo detrend infile outfile
I used
cdo trend infile afile bfile
and
cdo subtrend infile afile bfile outfile T
The documentation explains that detrend is equal to trend+subtrend and it's worked. But I always end up with this bug at only this 1980-09-12 day. Consequently, I cannot calculate EOFs or anything else. Would you have any idea on how to solve it?
| Antarctic Sea Ice.jpeg (64.8 KB) Antarctic Sea Ice.jpeg |
Replies (3)
RE: How to solve a white spot at the 1980-09-12 day in a NetCDF4 dataset which prevents me to operate anything - Added by Karin Meier-Fleischer 8 months ago
Hi Jose,
the problem is that we have neither the data nor the error message. And what is a 'normal viewer'? The only thing that can be said about your problem is that if there is a bug in a time step, it needs to be fixed.
RE: How to solve a white spot at the 1980-09-12 day in a NetCDF4 dataset which prevents me to operate anything - Added by José Carlos Barbosa Neto 8 months ago
Hi, Karin.
The dataset is a 1,5 GB file, so it's impossible to share it here. It has no problem with time step neither. I'm using the "Sea Ice Concentrations from Nimbus-7 SMMR and DMSP SSM/I-SSMIS Passive Microwave Data, Version 2" daily dataset and from 1978 to 1987 it displays the data each two days. The 1980-09-12 is an interpolated value and is shown with that huge white spot whenever I detrend this 1,5 GB file. I've tested another smaller dataset with this 1980-09-12 within and I've have no problem, it have no white spot. So I think it's not a data structure problem for this day.
In fact, CDO generates a detrended file (with this white spot) I can produce the anomaly with, but whenever I use the command for calculate EOFs the message error is
cdo eof (Abort): Missing values unsupported!
cdi error (cdf_close): NetCDF: HDF error
I think CDO is reading that spot as missing values too, but it does not recognize their structure, I don't know. Unfortunately, I could not discard this day, since I'm working with daily analysis.
RE: How to solve a white spot at the 1980-09-12 day in a NetCDF4 dataset which prevents me to operate anything - Added by Karin Meier-Fleischer 8 months ago
Your data contains missing values and the docu says about it in https://code.mpimet.mpg.de/projects/cdo/embedded/cdo.pdf#subsection.2.11.1:
Missing values are not fully supported. Support is only checked for non-changing masks of missing values in time. Although there still will be results, they are not trustworthy, and a warning will occur. In the latter case we suggest to replace missing values by 0 in infile.