Assessing the Skill of Coupled Atmosphere-Land-Ocean Climate Models

Lavers et al. (2009) conducted what they describe as "a careful analysis of the predictive skill of temperature and precipitation from eight seasonal climate forecast models" that were developed at various European climate centers. This they did by assessing the predictability of monthly temperature and precipitation "retrospective forecasts" or hindcasts, which were composed of multiple nine-month projections initialized during each month of the year over the period 1981-2001, comparing the projections against real-world air temperatures and precipitation values that were obtained, respectively, from ERA-40 reanalysis data and Global Precipitation Climatology Center data. In addition, they conducted a virtual-world analysis, where the output of one of the models was arbitrarily assumed to be the "truth," and where the average of the rest of the models was assumed to be the "predictor." So what did they find?

Lavers et al. report that in the virtual world of the climate models, there was quite good skill over the first two weeks of the forecast, when the spread of ensemble model members was small, but that there was a large drop off in predictive skill in the second 15-day period. Things were even worse in the real world, where they say the models had negligible skill over land at a 31-day lead time, which they described as being "a relatively short lead time in terms of seasonal climate prediction."

Based on these results, the three researchers conclude that given the real-world skill -- or lack thereof! -- demonstrated by the state-of-the-art models, "it appears that only through significant model improvements can useful long-lead forecasts be provided that would be useful for decision makers," a quest that they quite frankly state "may prove to be elusive." Such an assessment adds to concerns about why anyone should believe anything the models predict about climate that extends decades and centuries into the future, when they currently cannot provide a useful prediction more than a month in advance.