Project aims

The work-package’s aim is to enhance the capabilities of the verification tool (CES plug-in) which has been developed during the first phase of MiKlip. This comprises the introduction of new methods both on the satellite side and the dynamically oriented verification using classical variables. Satellite simulators will be developed to obtain virtual satellite observations (e.g. brightness temperatures, radar reflectivities) from the forecast system. The approach enables an evaluation in the instrument’s parameter space which reduces uncertainties on the side of the observational data. The work package also focuses on the development of probabilistic verification methods applicable to decadal predictions and  the analysis of dynamical processes in the coupled atmosphere-ocean system relevant for decadal predictions. An overall project aim is the preparation towards an operational verification of the MiKlip system.

Project structure

The work-package constitutes a joint project of DWD and MIUB (Meteorological Institute of the University of Bonn). DWD focuses on the provision and utilisation of satellite data. MIUB contributes by analysis of dynamical processes and the development and application of probabilistic evaluation methods. Satellite simulators and CDRs will be provided within the first phase of the project. Results of dynamical analysis will be provided to support the further development of the probabilistic evaluation methods. The preparation towards an operational verification covers the complete lifetime of the work-package.

Tasks of the project

The project tasks are (i) the provision of high quality satellite based data (mature CDRs), (ii) the development of a satellite simulator for microwave imagers, (iii) the analysis of circulation dependencies, (iv) event based conditional sampling,  (v) the introduction of ensemble Kernel dressing (EKD), (vi) hindcast verification and (vii) initial operationalization of the verification plug-in for the CES.

Deliverables

Satellite simulators for passive microwave imagers (SSM/I and SSMIS) will be delivered applicable to MPI-ESM and suited for hindcast evaluation. Moreover, the work-package will provide high quality observational data obtained from satellite measurements. These data constitute mature climate data records (CDRs) and will be provided in NetCDF (CMOR conform) for the CES. The verification plug-in for the CES will be upgraded by the introduction of Kernel dressing methods. Moreover, event based/conditional sampling methods for analysis of tropical and extratropical circulation patterns utilising satellite data will be incorporated. The satellite simulator and the verification-plugin will be used for hindcast evaluation. Moreover, an initial operational version of the verification plug-in for the CES will be provided.

Progress so far

Probabilistic evaluation methods are further developed and applied to assess the predictive skill of the MiKlip system. Furthermore, a satellite simulator has been developed for the Special Sensor Microwave Imager (SSM/I) and for the Special Sensor Microwave Imager and Sounder (SSMIS) utilising the CFMIP Observation Simulator Package (COSP). On the reference side the SSM/I & SSMIS Fundamental Climate Data Record (FCDR) provided by CM SAF (DOI: 10.5676/EUM_SAF_CM/FCDR_MWI/V003) is used.

The simulator is applied to the MiKlip II pre-operational hindcasts to evaluate the climatological and predictive skill of the MiKlip system focusing on different global water cycle components. The simulated brightness temperatures resemble the general structure and amplitude of the observations for multi-year averages. Probabilistic evaluation results covering the period from 1988 to 2014 indicate predictive skill for both lead year 1 and lead year 2-5 over tropical and sub-tropical ocean areas. Less predictive skill is found over the eastern Pacific north and south of the equator. Depending on the channel and lead years selected, distinct spatial patterns are identified. For the 22 GHz channel, which is sensitive to water vapour, probabilistic evaluation results are consistent with a classical evaluation approach using satellite retrieved water vapour from HOAPS4 (DOI: 10.5676/EUM_SAF_CM/HOAPS/V002) on the reference side.

The probabilistic evaluation methods have been applied to the series of MiKliP Phase 1 hindcast experiments B1-LR, B1-MR, prototype ORAs4, prototype GECCO, the MiKliP Phase 2 pre-operational hindcasts and the six experimental hindcast simulations performed by Module A. The central theoretical result is that for each ensemble the sharpness of the predictions based on an analysis of variance anova (analysis of variance; how tightly do all single ensemble members simulate the same signal) provides essential information for the forecast quality. The ensemble spread score ESS which is often taken as an independent skill measure can be written as function of the correlation between ensemble mean and observation and anova. It should be kept in mind that an ideal ESS equal to one can be achieved if there is no joint signal or no sharpness in the model ensemble. This happens because the ESS is a measure of reliability which is the balance between the forecast spread and the forecast uncertainty. Here an ensemble spread equal to climate variance goes along with zero correlation and an ESS of one. A second central results is the comparable probabilistic evaluation of continuous predictions and categorial ones based on an information entropy approach and allowing the definitions of generalised correlation and anova to specify skill and sharpness for both types of variables.