Project

General

Profile

Handling of satellite data from EUMETSAT

Added by Remi Montroty about 3 years ago

Dear all,

I've inherited a netCDF file from EUMETSAT satellite Metop (IASI L2 instrument).

It is a polar orbiting satellite and I've got one orbit of data. Can I use cdo to remap it to say a 10km global grid?

If so, what would the command be ?
Ultimately I'd like to use ml2pl command to remap from the variables on model levels (they have been interpolated to ECMWF 137 model hybrid levels) to pressure levels.

Something like this:

cdo ml2pl,plevels ifile.nc ofile.nc

However, Currently it skips the T & W variables (which are the ones of interest):

cdo ml2pl,100000,97500,95000,92500,90000,87500,85000,82500,80000,77500,75000,72500,70000,67500,65000,62500,60000,57500,55000,52500,50000,47500,45000,42500,40000,37500,35000,32500,30000,27500,25000,22500,20000,17500,15000,12500,10000,7500,5000,2500 IASI_PW3_02_M02_20180101053553Z_20180101071752Z_R_O_20201217175637Z_0100.nc test.nc
Warning (cdf_set_var): Inconsistent variable definition for latitude!
Warning (cdf_set_var): Inconsistent variable definition for longitude!
Warning (cdf_set_var): Inconsistent variable definition for SensingTime!
Warning (cdf_set_var): Inconsistent variable definition for pressure_levels!
Warning (cdf_read_ycoord): Unsupported array structure, skipped variable SensingTime_msec!
Warning (cdf_read_coordinates): Time varying grids unsupported, using grid at time step 1!
Warning (cdf_read_xcoord): Unsupported array structure, skipped variable T!
Warning (cdf_read_xcoord): Unsupported array structure, skipped variable W!
Warning (cdf_read_xcoord): Unsupported array structure, skipped variable FLG_AMSUBAD!
Warning (cdf_read_ycoord): Unsupported array structure, skipped variable FLG_INITIA!
Warning: No 3D variable with hybrid sigma pressure coordinate found!
cdo    ml2pl: Processed 18 variables over 765 timesteps [1.23s 82MB].

Not sure what I'd need to have a properly supported array structure.

Thanks for any help!

ps: I have provided a sample file of the current version of the NetCDF.
There might be some required changes to the structure to make it more cdo-compatible: any advice welcome!


Replies (3)

RE: Handling of satellite data from EUMETSAT - Added by Brendan DeTracey about 3 years ago

$ cdo -v verifygrid IASI_PW3_02_M02_20180101002657Z_20180101021152Z_R_O_20201217181451Z_0100.nc
 OpenMP:  num_procs=24  max_threads=1
Warning (cdf_set_var): Inconsistent variable definition for latitude!
Warning (cdf_set_var): Inconsistent variable definition for longitude!
Warning (cdf_set_var): Inconsistent variable definition for SensingTime!
Warning (cdf_set_var): Inconsistent variable definition for pressure_levels!
Warning (cdf_read_ycoord): Unsupported array structure, skipped variable SensingTime_msec!
Warning (cdf_read_coordinates): Time varying grids unsupported, using grid at time step 1!
Warning (cdf_read_xcoord): Unsupported array structure, skipped variable T!
Warning (cdf_read_xcoord): Unsupported array structure, skipped variable W!
Warning (cdf_read_xcoord): Unsupported array structure, skipped variable FLG_AMSUBAD!
Warning (cdf_read_ycoord): Unsupported array structure, skipped variable FLG_INITIA!

cdo    verifygrid (Abort): Grid coordinates missing!
Your file is not CF Metadata compliant (https://cfconventions.org/index.html). I would ask around or google whether anyone has done this conversion for this file type. If not, you will have to convert it yourself with generous use of the showattribute commands and setattribute commands (or NCO ncatted command). The CF Metadata page has links to compliance checkers.

RE: Handling of satellite data from EUMETSAT - Added by Karin Meier-Fleischer about 3 years ago

Hi Remi,
Brendan is right, the data is not CF conform. But you can use NCL to read the data, remap it to a global grid, and write the result o a netCDF file.
I'll try to find my IASI example and get back to you.

-Karin

RE: Handling of satellite data from EUMETSAT - Added by Karin Meier-Fleischer about 3 years ago

Regarding the pressure level question, the IASI file contains a variable called pressure_levels

float pressure_levels(scan_lines, pixels, vertical_levels) ;
        pressure_levels:_FillValue = 3.402823e+38f ;
        pressure_levels:coordinates = "latitude longitude pressure_levels SensingTime" ;
        pressure_levels:units = "hPa" ;
        pressure_levels:ancillary_variables = "QP" ;
        pressure_levels:standard_name = "air_pressure" ;
        pressure_levels:long_name = "Pressure" ;
        pressure_levels:description = "Atmospheric pressure profile [137 atmospheric levels + surface] (hPa)" ;
        pressure_levels:valid_min = 0.f ;
        pressure_levels:valid_max = 1050.f ;

You have to ask the EUMTESAT people how to deal with it.

I was able to read, remap, plot, and write the regridded data to a new netCDF file using an NCL script.
Therefore, I choose the variable T create the following example.

Level 1:

Level 138:

NCL script:

load "$NCARG_ROOT/lib/ncarg/nclscripts/esmf/ESMF_regridding.ncl" 

diri  = "$HOME/Downloads/" 
fname = "IASI_PW3_02_M02_20180101002657Z_20180101021152Z_R_O_20201217181451Z_0100.nc" 
odir  = "./" 
ofile =  str_get_field(fname,1,".")+"_remap_to_T63.nc" 

;-- read swath data
f = addfile(diri+fname,"r")

lat2d = f->latitude
lon2d = f->longitude
delete(lat2d@coordinates)
delete(lon2d@coordinates)

nlevels = dimsizes(f->T(0,0,:))

;-- variable at level index x

var = f->T(:, :, 0)
;var = f->T(:, :, nlevels-1)

var = where(var .gt. 400, var@_FillValue, var)
var@coordinates = "latitude longitude pressure_levels" 
var@lon2d = lon2d
var@lat2d = lat2d

;-- regridding options
Opt = True
Opt@SrcFileName  = "IASI_T_src_SCRIP.nc" 
Opt@DstFileName  = "IASI_T_dst_SCRIP.nc" 
Opt@WgtFileName  = "IASI_T_1x1deg_bilinear.nc" 
Opt@ForceOverwrite = True
Opt@SrcMask2D    = where(.not. ismissing(var),1,0)
Opt@DstGridType  = "1x1" 
Opt@DstTitle     = "World grid 1x1 degree resolution" 
Opt@DstLLCorner  = (/-89.75d, -179.75d /)
Opt@DstURCorner  = (/ 89.75d,  179.75d /) 
Opt@InterpMethod = "bilinear" 

var_regrid_b = ESMF_regrid(var,Opt)

;-- write regridded data to new netCDF file

system("rm -rf IASI_L2_regrid_bilinear_1x1_T_lev0.nc")
fout = addfile("IASI_L2_regrid_bilinear_1x1_T_lev0.nc", "c")
fout->T = var_regrid_b

;-- Plotting section
wks = gsn_open_wks("png","plot_regrid_IASI_L2")

res                      = True 
res@gsnFrame             = False
res@gsnLeftString        = "" 
res@gsnRightString       = "" 
res@gsnAddCyclic         = False

res@vpHeightF            = 0.38
res@vpXF                 = 0.14
res@vpYF                 = 0.95

res@cnLinesOn            = False
res@cnFillMode           = "RasterFill" 
res@cnLineLabelsOn       = False
res@cnFillOn             = True
res@cnFillPalette        = "BlAqGrYeOrReVi200" 
res@cnLevelSelectionMode = "ManualLevels" 
res@cnMinLevelValF       = 150.0
res@cnMaxLevelValF       = 320.0
res@cnLevelSpacingF      = 2.0   

res@mpFillOn             = True
res@mpOceanFillColor     = "gray" 
res@mpInlandWaterFillColor = "gray" 
res@mpLandFillColor      = "gray" 

res@lbLabelBarOn         = False
res@pmTickMarkDisplayMode = "Always" 

res@tiMainFontHeightF    = 0.012
res@tiMainOffsetYF       = -0.015
res@tiMainString         = var@long_name

plot = gsn_csm_contour_map(wks,var,res)     

res@vpYF                 = 0.468

res@tiMainString         = var@long_name + " - regrid 1x1 deg" 
res@lbLabelBarOn         = True
res@lbLabelFontHeightF   = 0.008
res@lbBoxMinorExtentF    = 0.1
res@pmLabelBarOrthogonalPosF = 0.01
res@pmLabelBarWidthF     = 0.7

plot_b = gsn_csm_contour_map(wks,var_regrid_b,res)     

frame(wks)
    (1-3/3)