Project

General

Profile

Unable to open .nc file (unknown error)

Added by Andrea Galletti 5 months ago

Hi, I got an nc file from a web repository.

the file itself is trusted, though i doubt it was properly built for cdo operation. The creator told me that he can sufficiently handle it with python xarray, however i'd like to stick to CDO for my ends.

i was initially trying to look through the file attributes etc, so i ran

$ cdo info 2011/ITSNOW_SWE_201101.nc
cdo info: Open failed on >2011/ITSNOW_SWE_201101.nc<
Unknown Error

can anyone help me troubleshooting this? attached you can find one example file (it's a snow cover in Italy, you can safely open it in qgis to check)

Thanks!

ITSNOW_SWE_201011.nc (17.2 MB) ITSNOW_SWE_201011.nc sample file for cdo use

Replies (8)

RE: Unable to open .nc file (unknown error) - Added by Karin Meier-Fleischer 5 months ago

Hi Andrea,

which CDO version do you use? CDO version >2.0.5 works:

cdo sinfon ITSNOW_SWE_201011.nc 
Warning (cdfScanVarAttr): NetCDF: Variable not found - >latitude<
Warning (cdfScanVarAttr): NetCDF: Variable not found - >longitude<
   File format : NetCDF4 zip
    -1 : Institut Source   T Steptype Levels Num    Points Num Dtype : Parameter name
     1 : unknown  https://doi v instant       1   1   6713874   1  F64z : SWE           
   Grid coordinates :
     1 : projection               : points=6713874 (2727x2462)
                          mapping : latitude_longitude
                        Longitude : 6.475217 to 20.26216 by 0.005057571 degrees
                         Latitude : 35.14584 to 47.59252 by 0.005057571 degrees
   Vertical coordinates :
     1 : surface                  : levels=1
   Time coordinate :
                             Time : 30 steps
     RefTime =  2010-11-01 00:00:00  Units = days  Calendar = proleptic_gregorian
  YYYY-MM-DD hh:mm:ss  YYYY-MM-DD hh:mm:ss  YYYY-MM-DD hh:mm:ss  YYYY-MM-DD hh:mm:ss
  2010-11-01 00:00:00  2010-11-02 00:00:00  2010-11-03 00:00:00  2010-11-04 00:00:00
  2010-11-05 00:00:00  2010-11-06 00:00:00  2010-11-07 00:00:00  2010-11-08 00:00:00
  2010-11-09 00:00:00  2010-11-10 00:00:00  2010-11-11 00:00:00  2010-11-12 00:00:00
  2010-11-13 00:00:00  2010-11-14 00:00:00  2010-11-15 00:00:00  2010-11-16 00:00:00
  2010-11-17 00:00:00  2010-11-18 00:00:00  2010-11-19 00:00:00  2010-11-20 00:00:00
  2010-11-21 00:00:00  2010-11-22 00:00:00  2010-11-23 00:00:00  2010-11-24 00:00:00
  2010-11-25 00:00:00  2010-11-26 00:00:00  2010-11-27 00:00:00  2010-11-28 00:00:00
  2010-11-29 00:00:00  2010-11-30 00:00:00

To get rid of the warnings you can use NCO's ncatted tool to delete the unecessary coordinates attribute from the variable SWE:

ncatted -O -a coordinates,SWE,d,, ITSNOW_SWE_201011.nc
cdo sinfon ITSNOW_SWE_201011.nc
   File format : NetCDF4 zip
    -1 : Institut Source   T Steptype Levels Num    Points Num Dtype : Parameter name
     1 : unknown  https://doi v instant       1   1   6713874   1  F64z : SWE           
   Grid coordinates :
     1 : projection               : points=6713874 (2727x2462)
                          mapping : latitude_longitude
                        Longitude : 6.475217 to 20.26216 by 0.005057571 degrees
                         Latitude : 35.14584 to 47.59252 by 0.005057571 degrees
   Vertical coordinates :
     1 : surface                  : levels=1
   Time coordinate :
                             Time : 30 steps
     RefTime =  2010-11-01 00:00:00  Units = days  Calendar = proleptic_gregorian
  YYYY-MM-DD hh:mm:ss  YYYY-MM-DD hh:mm:ss  YYYY-MM-DD hh:mm:ss  YYYY-MM-DD hh:mm:ss
  2010-11-01 00:00:00  2010-11-02 00:00:00  2010-11-03 00:00:00  2010-11-04 00:00:00
  2010-11-05 00:00:00  2010-11-06 00:00:00  2010-11-07 00:00:00  2010-11-08 00:00:00
  2010-11-09 00:00:00  2010-11-10 00:00:00  2010-11-11 00:00:00  2010-11-12 00:00:00
  2010-11-13 00:00:00  2010-11-14 00:00:00  2010-11-15 00:00:00  2010-11-16 00:00:00
  2010-11-17 00:00:00  2010-11-18 00:00:00  2010-11-19 00:00:00  2010-11-20 00:00:00
  2010-11-21 00:00:00  2010-11-22 00:00:00  2010-11-23 00:00:00  2010-11-24 00:00:00
  2010-11-25 00:00:00  2010-11-26 00:00:00  2010-11-27 00:00:00  2010-11-28 00:00:00
  2010-11-29 00:00:00  2010-11-30 00:00:00

RE: Unable to open .nc file (unknown error) - Added by Andrea Galletti 5 months ago

Wow that is a lot clearer.

So, I'm afraid I am using the 1.9.x version provided by cygwin..

I downloaded the latest binaries, untarred them, but then didn't really do much else.

Then I downloaded cygwin, some other CDO.exe and another file I could find elsewhere here, copied them in the PATH directory and finally launched my cdo command as per OP.

How can I make use of the binaries I downloaded? Can I build different cdo.exe from there? How to?

Obviously I'm on windows, 11.

Thanks!

RE: Unable to open .nc file (unknown error) - Added by Karin Meier-Fleischer 5 months ago

That is a really old version. Have a look at the installation page https://code.mpimet.mpg.de/projects/cdo/wiki/Win32 (scroll down to Windows 10) or you can install CDO via conda in the Windows PowerShell.

RE: Unable to open .nc file (unknown error) - Added by Andrea Galletti 5 months ago

I see, thank you.

I will try conda in case I don't succeed with cygwin.

But I'm not sure I got what is written in the page you linked (I went through it more than once...). Can I use the binaries I downloaded in cygwin or does that bind me to the 1.9.x version?

Thanks for your patience:)

RE: Unable to open .nc file (unknown error) - Added by Karin Meier-Fleischer 5 months ago

I don't know which version is provided by cygwin. I recommend to use conda.

RE: Unable to open .nc file (unknown error) - Added by Estanislao Gavilan 5 months ago

Dear Andrea,

what you are experiencing with Cygwin is normal. Also, as Karin said that version is extremly old. I only use cygwin for simple things such as timmean or showdate. I also recommend install cdo via conda in a linux environment. Remember cdo is only fully supported in linux and macOS environment. If you want to use it in your windows, you need to install a ubuntu virtual machine in WSL2 or virtual box. I hope it helps

Kind regards,

Estanislao

RE: Unable to open .nc file (unknown error) - Added by Andrea Galletti 5 months ago

ok, i was still having troubles in conda but the 2.0.x version provided in WSL gave me the same output as Karin. Thanks everybody!

I now would like to perform some simple operations on these files, before actually processing them. can anyone show me the correct way of going through them?

Final goal: have a time-merged file over a subset of the domain

i'd like to
1) print out a .nc file that i can open on the likes of QGIS to visually inspect the indexes of my grid points and understand the extents of my subset
2) use a command to subset all files based on the same index range (i can work on a rectangular subset)
3) reproject the subsetted files to UTM 32N
4) time-merge them (is it ok if they skip the march-august window every year? will the time be consistently in increasing order anyways or will it produce a continuous time axis?

i plan on using the merged-subsetted-reprojected data on R for further analysis, i am also listening to suggestions for optimized computation

Again, thanks!

RE: Unable to open .nc file (unknown error) - Added by Estanislao Gavilan 5 months ago

Hi Andrea,

I will try to reply one by one.

1) I never used QGIS, but as long as your QGIS follows the cf-convention it should not be problem
2) You can select region by index or coordinates with selbox (see Documentation 2.3.5 )

cdo sellonlatbox,lon1,lon2,lat1,lat2  infile outfile
cdo selindexbox,idx1,idx2,idy1,idy2 infile outfile

3) You can reproject your data using the remap commands (see Documentation 2.12 Interpolation). CDO has a set of predefined grids, but I am not sure if the UTM 32 grid is one of them. You can easily specify your grid in a grid.txt like the follow

gridtype = lonlat
gridsize = 1800
xsize = 60
ysize = 30
xfirst = -177
xinc = 6
yfirst = -87
yinc = 6

and then,

cdo remapbil,grid.txt infile outfile

4) You can merge your data using the command cat or mergetime (see Documentation 2.2.1 APPLY - Apply operators) and select/remove years/months/days/dates with the command seltime (see Documentation 2.3.4).

cdo -cat 'infile1 infile2 infile3 infile4' outfile
cdo -selmonth,1,2,9,10,11,12 infile outfile

There are many examples on this forum. You can use the search bar to find the ones that are more similar to your problem. Then, we can keep helping you if you struggle with the code or errors

Kind regards,

Estanislao

    (1-8/8)