The main objectives of Module E are the quantification and documentation of (systematic) errors of the MiKlip model system, the validation of the decadal predictions, the estimation of their skill and the identification of predictable processes and scales in space and time.

To harmonising and synthesizing the activities across all projects, Module E coordination organised several hands-on workshops on forecast verification for decadal predictions additionally to the regular Module meetings. Furthermore, a common set of guidelines for verifying decadal predictions have been agreed upon and recorded in the MiKlip wiki, including bias correction, spatial and temporal scales, reference data sets, as well as scores to use. To promote the use of these guidelines among projects, Module E coordination participated in the development of a component for the evaluation system (INTEGRATION, Module D) which performs a set of standard evaluations complying with these guidelines. Many projects within Module E and other MiKlip modules have started to use this component for their analyses. As a consequence, verification within Module E and beyond has been streamlined; the strategies are well-known across MiKlip projects and as plots are similar, their message is conveyed easily and efficiently. Overall, verification within MiKlip has become transparent, reproducible and a common understanding of results has been generated across projects. Results from individual projects from Module E and other MiKlip Modules appear in a special issue of the Meteorologische Zeitschrift (MetZ) entitled Verification and process oriented validation of the MiKlip decadal prediction system.
In the following, we group the results from the various individual projects such that different aspects of the same process or related processes are combined.

Large-scale features

Extra-tropical cyclones have been identified and tracked in the MiKlip prediction system (EnsDiVal). Probabilistic predictions for three categories of cyclone frequency (below/normal/above) have been derived from the various ensemble hindcast experiments and verified against the 20th Century Reanalysis, NCEP/NCAR Reanalysis I and ECMWF reanalyses (ERA40/ERA-Interim). Areas with significant positive skill with respect to the climatological forecasts and the uninitialised historical runs vary with the considered model setups, see Fig. 1. However, hindcasts based on full-field initialisation turned out to be superior to anomaly initialisation for all lead times. Furthermore, the MPI-ESM-LR (like all general circulation models of comparable resolution) exhibits a spatial bias regarding both Northern Hemisphere storm tracks (too zonal over the North Atlantic, northward shifted over North Pacific).

Vertical profiles and cross sections

A comparison of quality controlled and homogenised radiosonde observations to model results shed light on the characteristics of temperature and humidity vertical profiles (MOSQUITO). Time series of stratospheric temperatures indicate that the cooling trend is captured in baseline0-LR and baseline1-LR, interannual variations are slightly better reproduced in the baseline1 setup. However, the cooling following major volcanic events is poorly represented, even though volcanic aerosols are prescribed according to the CMIP5 protocol. The correlation for hindcasts with respect to radiosonde data increases as soon as the homogenised series are used, which is in line with the finding that vertical temperature profiles obtained from radio occultation (GeoClim) yield positive skill with respect to the climatological forecast. Skill improves from baseline0-LR to baseline1-LR and even more than from baseline 0-LR to baseline 1-MR. VeCAP analysed near surface variables and additionally upper air and deep ocean fields as well as vertical cross sections of zonal means. In Fig.3, a measure for reliability (reliable, potentially useful and not reliable predictions) and accuracy (MSESS) are shown for annual mean vertical cross sections of geopotential heights (from 1979 to 2012) with ERA-Interim as reference. In the subtropical regions the predictions are reliable/potentially useful and show also high MSESS-values. Missing skill in the tropics could be traced back to large-scale developing structures in both baseline1-LR and -MR, which can be confirmed by applying the Continuous Ranked Probability Skill Score (CRPSS). For higher levels and longer lead times the gain of the increased model resolution (MR compared to LR), which can be seen in the tropical region, is not robust under bootstrappig.

Clouds and precipitation

VESPA investigated the link between West African Monsoon (WAM) and the SST patterns in the hindcast experiments based on SST indices and the standardised precipitation index (SPI) for regions dominated by the WAM. While in principle the link to Niño3.4 could be confirmed, with lower strength, links to other SST patterns could not be found. This applies similarly for both, baseline0-LR and baseline1-LR. The lack in representing the WAM could be traced back to two phenomena: i) AEW activity being strongly overestimated – which is in line with the results of VeCAP and VADY– and ii) the ITCZ being too far south during Monsoon periods. VeCAP shows a potential predictability of total cloud cover over the eastern North Atlantic derived from satellite cloud parameters for baseline1-LR. A similar potential predictability appears for the tropical warm pool in baseline0-LR, which can be attributed to shortcomings of the associated initialisation strategy.

Extremes

For various severe weather phenomena, a probabilistic model was build based on observations which describes the probability of occurrence dependent on relevant predictors such as vertical wind shear and Convective Available Potential Energy (CAPE) (STEPCLIM). Transferring the model to ERA-Interim and to hindcasts from baseline1-LR and baseline1-MR yields an overestimation of severe weather events in the hindcasts compared to ERA-Interim. This could be traced back to a steeper lapse rate and increased humidity in the lower troposphere, and consequently higher CAPE in baseline1-LR and MR. The latter is in line with MOSQUITO reporting a negative temperature, and a positive humidity bias in the troposphere and increased severe weather indices. Furthermore, according to the probabilistic model, STEPCLIM found more hail over orographic terrain, e.g. Turkey and Spain, in the baseline1 hindcasts than for ERA-Interim, which features local minima in these locations. Particularly over Europe VESPA localized too small dynamic instabilities in the initialized and uninitialised runs compared to ERA-Interim using the Dynamic State Index (DSI). Nevertheless, a steady improvement from baseline0 to prototype has been revealed for every lead year. Moreover, the amount of precipitation turned out to be overestimated compared to ERA-Interim, whereas prototype outperforms all other model setups. Since the DSI is an indicator for non-balanced processes including precipitation, there is a discrepancy between overestimated precipitation and underestimated DSI. A possible explanation might be the parameterization method. Based upon a set of extended hindcasts (described below) DroughtClip could reproduce the observed trend in drought-affected area in the US. Important anomalies, such as the “Dust-Bowl” during the 1930ies could however not be captured.

Ocean

A comparison of the decadal prediction system with the high resolution ocean reanalysis MIMOC has been carried out by OceanObs. Global mixed layer temperatures and salinities match relatively well with the MIMOC climatological cycle across different model setups with an exception of the North Atlantic, where the characteristics of the North Atlantic Current and the North Atlantic mixed layer depth are not well reproduced.

Sea-ice

Arctic summer sea ice extent could be shown to be well represented in the MPI-ESM, both in terms of the long-term trend and the seasonal cycle (ClimVal). Shortcomings are most prominent in the Antarctic where a significant negative bias with respect to the ClimVal sea-ice data set exists. This could be established across all model development stages, resolutions and initialization strategies.
Extended hindcasts and sensitivity studies
For the first time, hindcast experiments dating back to 1901 have been carried out by DroughtClip. Three assimilation runs were set up based on a reconstructed ocean state with MPIOM. They constitute the basis for three hindcasts completed with annual initialisation, starting from 1900 to 2009. These hindcasts' lead-year 2 to 5 predictions of the North-Atlantic SST outperform the uninitialised runs in their ability to reproduce the higher temperatures in 1930s and 1940s, as well as the cold phase during the 1970s (see Fig. 4). This closer correspondence is accompanied by higher correlations of the initialised predictions in the detrended series. A comparison between the extended hindcasts and the subset for the time period from 1960 to 2010 reveals an increase of surface temperature correlations for the extended period. This is particularly evident in the North Atlantic, where multi-decadal variability plays a large role, with detrended time series showing higher correlations for the extended compared to the short period. A sensitivity study with respect to cold and warm AMV phases has been set up by VESPA with AMV cold/warm anomalies derived from HadISST superimposed upon the monthly model climatology. This SST data set was used to drive the atmospheric component of the MiKlip prediction system.