Project

General

Profile

News

Redmine User Information: Redmine Upgrade coming soon

Added by Jan Sellmann over 10 years ago

The Redmine System on this platform is going to be upgraded soon, please expect a service interruption due to the maintenance in late November.
A condensed list of fixes and features available with the new version is available at Redmine-New-Features.

As soon as the final upgrade date is set, we'll let you know through this News system.

CDO: Version 1.6.2 released

Added by Uwe Schulzweida over 10 years ago

New features:
  • select: added support for key >timestep_of_year<
  • mastrfu: added missing value support
  • splitmon: added optional parameter to set the format string for the month
Fixed bugs:
  • selyear: wrong result for negative years [Bug #3836]
  • eca_gsl: start date of growing season is wrong if the length of growing season is zero

CDI: Version 1.6.2 released

Added by Uwe Schulzweida over 10 years ago

New features:
  • Added support for netCDF CF time attribute: climatology [Feature #4092]
  • Added CDI function zaxisDefNlevRef() and zaxisInqNlevRef() [Feature #4117]
  • Made gridtype GRID_REFERENCE part of GRID_UNSTRUCTED
  • GRIB2: added support for level type Lake Bottom (grib level type 162)
  • GRIB2: added support for level type Bottom Of Sediment Layer (grib level type 163)
  • GRIB2: added support for level type Bottom Of Thermally Active Sediment Layer (grib level type 164)
  • GRIB2: added support for level type Bottom Of Sediment Layer Penetrated By Thermal Wave (grib level type 165)
  • GRIB2: added support for level type Mixing Layer (grib level type 166)
Fixed bugs:
  • netCDF: wrong result if type of data and type of attribute valid_range differ [Bug #3727]

Redmine User Information: Password Changes for ZMAW users

Added by Jan Sellmann over 10 years ago

ZMAW-Users only:

From Tuesday, November 12th 2013 on the password for your ldap account on this site will be changed to your login password, the old "e-mail"-password won't work anymore. Please see Rainer Weigles e-mail to the zmaw-all mailinglist for details. In case of problems please contact .

Integration: NEWS from INTEGRATION - using&developing

Added by Christopher Kadow over 10 years ago

NEWS from 19.07.2013

Many projects inside of MiKlip joined the Evaluation System and the MiKlip Server by using it, brought in analyses (MesoTel, EnsDiVal) and data (VADY, MesoTel).

The Basic User Guide (BUG) by INTEGRATION with an overview of what has been developed or plugged in and how2use and load the evaluation system you can find here:

https://code.zmaw.de/projects/miklip-d-integration/wiki/

Always in mind, working on two aspects: using&developing , today we want to pick up two areas of your interest.

SOLR_SEARCH

SOLR_SEARCH

The MiKlip Server and the mounted DKRZ ESGF node combine a huge database with direct (!No need to download all data from the ESGF or from slow band archives!) access to over 50 model systems and 2 million files. That is why standardization is so important, just to find everything and stay comparable with the international community. INTEGRATION combined (linked) the different areas to one

/miklip/integration/data4miklip

and developed the search tool "find_files", both already presented at the MiKlip Status-Seminar 2013. But any normal search is too slow, to run through that amount of data.

Therefore INTEGRATION developed the new search system "solr_search" for the ongoing evaluation system and scientists working at the MiKlip server. It has the same syntax as "find_files".

"solr_search" is a very fast searching solution with a user friendly 'tab' usability.

Looking for the BASELINE1 runs?

[b324031@miklip04 ~]$ module load evaluation_system

[b324031@miklip04 ~]$ solr_search ->PRESS TABULATOR
cmor_table=      ensemble=        institute=       project=         time_frequency=  
data_type=       experiment=      model=           realm=           variable=

[b324031@miklip04 ~]$ solr_search data_type= ->PRESS TABULATOR
baseline0     baseline1     cmip5         observations  reanalysis

[b324031@miklip04 ~]$ solr_search data_type=baseline1 ->PRESS TABULATOR
cmor_table=      experiment=      realm=           variable=        
ensemble=        model=           time_frequency=

[b324031@miklip04 ~]$ solr_search data_type=baseline1 model=mpi ->PRESS TABULATOR
mpi-esm-lr  mpi-esm-mr  

[b324031@miklip04 ~]$ solr_search data_type=baseline1 model=mpi-esm-lr ->PRESS TABULATOR
cmor_table=      ensemble=        realm=           time_frequency=  variable=      

[b324031@miklip04 ~]$ solr_search data_type=baseline1 model=mpi-esm-lr variable= ->PRESS TABULATOR
Display all 100 possibilities? (y or n)
areacella      clivi          dissoc         fddtdisi       hfls           intpn2         no3
areacello      clt            dms            ffire          hfsithermds    intpp          npp
baresoilfrac   clw            dpco2          fgco2          hfss           lai            o2
basin          clwvi          dpo2           fgdms          hfx            landcoverfrac  o2min
bfe            co2mass        epc100         fgo2           hfxba          masso          omldamax
bsi            co3            epcalc100      fluc           hfxdiff        mfo            omlmax
burntarea      co3satcalc     epfe100        frc            hfy            mlotst         orog
c3pftfrac      cropfrac       epsi100        frfe           hfyba          mlotstsq       pasturefrac
c4pftfrac      csoil          evspsbl        frn            hfydiff        mrro           pbfe
calc           cveg           fco2nat        fsfe           hur            mrros          pbo...
cct            dcalc          fddtalk        fveglitter     hus            mrso           
chl            deptho         fddtdic        gpp            intdic         mrsofc         
cl             detoc          fddtdife       grassfrac      intpbfe        msftbarot      
cli            dfe            fddtdin        graz           intpbsi        msftmyz        
clitter        dissic         fddtdip        hfds           intpcalcite    nbp

etc etc etc

Looking for models to compare to your results in temperature?

solr_search variable=ta model= ->PRESS TABULATOR
access1-0        cesm1-bgc        ec-earth         giss-e2-h-cc     ipsl-cm5a-lr     mpi-esm-p
access1-3        cesm1-cam5       fio-esm          giss-e2-r        ipsl-cm5a-mr     mri-agcm3-2h
bcc-csm1-1       cesm1-fastchem   geos-5           giss-e2-r-cc     ipsl-cm5b-lr     mri-cgcm3
bnu-esm          cesm1-waccm      gfdl-cm3         hadcm3           miroc4h          noresm1-m
canam4           cmcc-cesm        gfdl-esm2g       hadgem2-a        miroc5           noresm1-me
cancm4           cmcc-cm          gfdl-esm2m       hadgem2-ao       miroc-esm        
canesm2          cmcc-cms         gfdl-hiram-c180  hadgem2-cc       miroc-esm-chem   
ccsm4            cnrm-cm5         gfdl-hiram-c360  hadgem2-es       mpi-esm-lr       
cdas             csiro-mk3-6-0    giss-e2-h        inmcm4           mpi-esm-mr       

etc etc etc

Do we have also observations for temperature?

[b324031@miklip04 ~]$ solr_search data_type=observations variable=ta ->PRESS ENTER
/miklip/integration/data4miklip/observations/atmos/ta/mon/grid/NASA-JPL/MLS/v20111025/ta_MLS_L3_v03-3x_200408-201012.nc
/miklip/integration/data4miklip/observations/atmos/ta/mon/grid/NASA-JPL/AIRS/v20110608/ta_AIRS_L3_RetStd-v5_200209-201105.nc

SOLR_SEARCH can be used for developing analyses getting direct access to datasets using their meta data and it makes the ensemble handling in CMOR/CMIP5 datasets much easier! It will be also the basement of the hybrid "shell/web/operational" solution of the evaluation system, to assure a fast and effective use of the evaluation system and the MiKlip server. The system is in the beta phase and ready to be filled up with the regional data as well as the next general MiKlip runs.

More and detailed informations:
https://code.zmaw.de/projects/miklip-d-integration/wiki/Solr_search

PROJECTDATA

Our project combined different areas of data in

/miklip/integration/data4miklip

This is about data not belonging to the general runs of MiKlip. Where could you put them to compare them to general MiKlip data?

We set up a projectdata directory:

/miklip/integration/data4miklip/projectdata

If you have your CMORized data e.g. in your scratch, send us an email, then we link it to the system.

What does CMORized structure mean?

Let's have a look at user data from MESOTEL Module C - b324034

/miklip/integration/data4miklip/projectdata/b324034

This is a link b324034 -> /scratch/b324034/archive/CMIP5/ You see the user decided to keep the standard CMIP5 CMOR settings, to test his data, therefore just use the switch in the MPI-ESM for postproduction.

When we go deeper into the structure we find e.g.

/miklip/integration/data4miklip/projectdata/b324034/output/MPI-M/MPI-ESM-LR/decadal2000/mon/atmos/tas/r1i1p2/tas_Amon_MPI-ESM-LR_decadal2000_r1i1p2_200101-201012.nc

You see we need a specific directory structure to put in. Of course you already had a look into CMOR :) If not, here is a short description what you need in your data structure for linking in and the example you see above.

EXAMPLE:

b324034/output/MPI-M/MPI-ESM-LR/decadal2000/mon/atmos/tas/r1i1p2/tas_Amon_MPI-ESM-LR_decadal2000_r1i1p2_200101-201012.nc

DIRECTORY:

project/product/institute/model/experiment/time_frequency/realm/variable/ensemble/

FILE:

"variable"_"cmor_table"_"model"_"experiment"_"ensemble"_"start_time"-"end_time".nc

And now the different stories coming together as one!

If you consider solr_search now and look for your data, just do:

solr_search project=b324034 experiment=decadal2000 variable= -> PRESS TAB
Display all 100 possibilities? (y or n) -> n
solr_search project=b324034 experiment=decadal2000 variable=tas time_frequency=mon -> PRESS ENTER
/miklip/integration/data4miklip/projectdata/b324034/output/MPI-M/MPI-ESM-LR/decadal2000/mon/atmos/tas/r1i1p2/tas_Amon_MPI-ESM-LR_decadal2000_r1i1p2_200101-201012.nc

And that means you can ask solr_search for data to compare to, here baseline1-LR!

solr_search project=b324034 project=baseline1 experiment=dec*2000 variable=tas time_frequency=mon model=mpi-esm-lr
/miklip/integration/data4miklip/model/baseline1/output/MPI-M/MPI-ESM-LR/decs4e2000/mon/atmos/tas/r9i1p1/tas_Amon_MPI-ESM-LR_decs4e2000_r9i1p1_200101-201012.nc
/miklip/integration/data4miklip/model/baseline1/output/MPI-M/MPI-ESM-LR/decs4e2000/mon/atmos/tas/r8i1p1/tas_Amon_MPI-ESM-LR_decs4e2000_r8i1p1_200101-201012.nc
/miklip/integration/data4miklip/model/baseline1/output/MPI-M/MPI-ESM-LR/decs4e2000/mon/atmos/tas/r7i1p1/tas_Amon_MPI-ESM-LR_decs4e2000_r7i1p1_200101-201012.nc
/miklip/integration/data4miklip/model/baseline1/output/MPI-M/MPI-ESM-LR/decs4e2000/mon/atmos/tas/r6i1p1/tas_Amon_MPI-ESM-LR_decs4e2000_r6i1p1_200101-201012.nc
/miklip/integration/data4miklip/model/baseline1/output/MPI-M/MPI-ESM-LR/decs4e2000/mon/atmos/tas/r5i1p1/tas_Amon_MPI-ESM-LR_decs4e2000_r5i1p1_200101-201012.nc
/miklip/integration/data4miklip/model/baseline1/output/MPI-M/MPI-ESM-LR/decs4e2000/mon/atmos/tas/r4i1p1/tas_Amon_MPI-ESM-LR_decs4e2000_r4i1p1_200101-201012.nc
/miklip/integration/data4miklip/model/baseline1/output/MPI-M/MPI-ESM-LR/decs4e2000/mon/atmos/tas/r3i1p1/tas_Amon_MPI-ESM-LR_decs4e2000_r3i1p1_200101-201012.nc
/miklip/integration/data4miklip/model/baseline1/output/MPI-M/MPI-ESM-LR/decs4e2000/mon/atmos/tas/r2i1p1/tas_Amon_MPI-ESM-LR_decs4e2000_r2i1p1_200101-201012.nc
/miklip/integration/data4miklip/model/baseline1/output/MPI-M/MPI-ESM-LR/decs4e2000/mon/atmos/tas/r1i1p1/tas_Amon_MPI-ESM-LR_decs4e2000_r1i1p1_200101-201012.nc
/miklip/integration/data4miklip/model/baseline1/output/MPI-M/MPI-ESM-LR/decs4e2000/mon/atmos/tas/r10i1p1/tas_Amon_MPI-ESM-LR_decs4e2000_r10i1p1_200101-201012.nc
/miklip/integration/data4miklip/projectdata/b324034/output/MPI-M/MPI-ESM-LR/decadal2000/mon/atmos/tas/r1i1p2/tas_Amon_MPI-ESM-LR_decadal2000_r1i1p2_200101-201012.nc

If you want to put in other data, like reanalysis to test something you can of course use this too. E.g.:

b324034/reanalysis/ECMWF/IFS/ERAINT/mon/atmos/tas/r1i1p1/tas_Amon_reanalysis_ERAINT_r1i1p1_197901-201212.nc

For help getting your specific data CMORized, please have a look into:

/miklip/integration/data_management/standardization_examples

There you find 1 example for decadal, historical, reanalysis and observation standardization.

This is the idea of bringing data together instead of changing the tools for different datasets. All ongoing projects developing analyses, will use this structure. If we can handle that in MiKlip, we will have a fast, effective, growing, and open evaluation system.

Please contact us, if you have any question or advice.

Christopher Kadow -
and
Sebastian Illing -

CDO: Version 1.6.1 released

Added by Uwe Schulzweida over 10 years ago

New features:
  • support of blanks in filenames and parameter
Changed operators:
  • gradsdes: added support for netCDF files
  • Info: add chunking information of netcdf files (only with verbose output) [Feature #3489]
  • select: added support for key >hour<
Fixed bugs:
  • fldcor: check missing value of 2. input file
  • enscrps: wrong result since CDO version 1.5.6 [Bug #3403]
  • selmon: month not found for negative years [Bug #3439]
  • shifttime: wrong result for negative hours and days [Bug #3440]
  • inttime: removes last time step [Bug #3611]

Redmine User Information: Important info on email notification

Added by Karl-Hermann Wieners over 10 years ago

Dear user of the ZMAW Redmine Service,

due to users' demand, and in our endless efforts to make our Redmine system as useful as possible, we would like to re-configure the system to allow email notification not only for changes in Issues (as it is now), but also for News, Documents, Wikis, Forums, and Files. This means that you may e.g. go to your favourite Forum to click the 'watch' button and get an email notification whenever someone posts a new message.

While this allows for more flexibility in using Redmine for your projects, we definitely want to avoid 'Redmine spamming' for those of you who prefer checking their favourite pages themselves, or use the Atom feeds to stay up-to-date.

So we would like you to take some minutes to read the following paragraphs about the potential impact of this change for your daily work.

The change will become effective on Friday, February 15th, at noon (UTC).

Of course, if you have further questions or feedback, feel free to contact us at any time (but only expect us to answer during German office hours...)

Best regards, Your ZMAW Redmine Administration Team

Attachment: Notification of Events in ZMAW's Redmine

CDO: Version 1.6.0 released

Added by Uwe Schulzweida about 11 years ago

New operators:
  • select: Select fields from an unlimited number of input files
  • mergegrid: Merge horizontal grids
  • yearmonmean: yearly mean from monthly data
  • duplicate: Duplicates a dataset
  • adisit: Potential temperature to in-situ temperature
  • rhopot: Calculates potential density
Changed operators:
  • setcalendar: changed CDO calendar names to CF calendar names (Feature #3123)
    (standard, proleptic_gregorian, 360_day, 365_day, 366_day)
  • masklonlatbox: added support for curvilinear grids
  • diff: print only records that differ
Fixed bugs:
  • sellonlatbox: wrong result with overlapped lonlatbox on curvilinear grids
  • ensrkhisttime: fixed memory fault
  • expr: wrong result for operation var1/var2 where var2 = 0
  • Runstat: added support for time bounds (Bug #3127)
  • merge: uses size of the first input file for the output buffer
(71-80/128)

Also available in: Atom