Wintest Analysis Essay

By Larry Kummer, from the Fabius Maximus website

Summary; Public policy about climate change has become politicized and gridlocked after 26 years of large-scale advocacy. We cannot even prepare for a repeat of past extreme weather. We can whine and bicker about who to blame. Or we can find ways to restart the debate. Here is the next of a series about the latter path, for anyone interested in walking it. Climate scientists can take an easy and potentially powerful step to build public confidence: re-run the climate models from the first 3 IPCC reports with actual data (from their future): how well did they predict global temperatures?

Trust can trump Uncertainty.”
— Presentation by Leonard A Smith (Prof of Statistics, LSE), 6 February 2014.

The most important graph from the IPCC’s AR5

Figure 1.4 from p131 of AR5: the observed global surface temperature anomaly relative to 1961–1990 in °C compared with the range of projections from the previous IPCC assessments. Click to enlarge.

Why the most important graph doesn’t convince the public

Last week I posted What climate scientists did wrong and why the massive climate change campaign has failed. After 26 years, one of the largest longest campaigns to influence public policy has failed to gain the support of Americans, with climate change ranking near the bottom of people’s concerns. It described the obvious reason: they failed to meet the public’s expectations for behavior of scientists warning about a global threat (i.e., a basic public relations mistake).

Let’s discuss what scientists can do to restart the debate. Let’s start with the big step: show that climate models have successfully predicted future global temperatures with reasonable accuracy.

This spaghetti graph — probably the most-cited data from the IPCC’s reports — illustrates one reason for lack of sufficient public support in America. It shows the forecasts of models run in previous IPCC reports vs. actual subsequent temperatures, with the forecasts run under various scenarios of emissions and their baselines updated. First, Edward Tufte probably would laugh at this The Visual Display of Quantitative Information — too much packed into one graph, the equivalent of a PowerPoint slide with 15 bullet points.

But there’s a more important weakness. We want to know how well the models work. That is, how well each forecast if run with a correct scenario (i.e., actual future emissions, since we’re uninterested here in predicting emissions, just temperatures).

The big step: prove climate models have made successful predictions

“A genuine expert can always foretell a thing that is 500 years away easier than he can a thing that’s only 500 seconds off.”
— From Mark Twain’s A Connecticut Yankee in King Arthur’s Court.

A massive body of research describes how to validate climate models (see below), most stating that they must use “hindcasts” (predicting the past) because we do not know the temperature of future decades. Few sensible people trust hindcasts, with their ability to be (even inadvertently) tuned to work (that’s why scientists use double-blind testing for drugs where possible).

But now we know the future — the future of models run in past IPCC reports — and can test their predictive ability.

Karl Popper believed that predictions were the gold standard for testing scientific theories. The public also believes this. Countless films and TV shows focus on the moment in which scientists test their theory to see if the result matches their prediction. Climate scientists can run such tests today for global surface temperatures. This could be evidence on a scale greater than anything else they’ve done.

Testing the climate models used by the IPCC

“Probably {scientists’} most deeply held values concern predictions: they should be accurate; quantitative predictions are preferable to qualitative ones; whatever the margin of permissible error, it should be consistently satisfied in a given field; and so on.”
— Thomas Kuhn in The Structure of Scientific Revolutions (1962).

The IPCC’s scientists run projections. AR5 describes these as “the simulated response of the climate system to a scenario of future emission or concentration of greenhouse gases and aerosols … distinguished from climate predictions by their dependence on the emission/concentration/radiative forcing scenario used…”. The models don’t predict CO2 emissions, which are an input to the models.

So they should run the models as they were when originally run for the IPCC in the First Assessment Report (FAR, 1990), in the Second (SAR, 1995), and the Third (TAR, 2001). Run them using actual emissions as inputs and with no changes of the algorithms, baselines, etc. How accurately will the models’ output match the actual global average surface temperatures?

Of course, the results would not be a simple pass/fail. Such a test would provide the basis for more sophisticated tests. Judith Curry (Prof Atmospheric Science, GA Inst Tech) explains here:

Comparing the model temperature anomalies with observed temperature anomalies, particularly over relatively short periods, is complicated by the acknowledgement that climate models do not simulate the timing of ENSO and other modes of natural internal variability; further the underlying trends might be different. Hence, it is difficult to make an objective choice for matching up the observations and model simulations. Different strategies have been tried… matching the models and observations in different ways can give different spins on the comparison.

On the other hand, we now have respectably long histories since publication of the early IPCC reports: 25, 20, and 15 years. These are not short periods, even for climate change. Models that cannot successfully predict over such periods require more trust than many people have when it comes to spending trillions of dollars — or even making drastic revisions to our economic system (as Naomi Klein and the Pope advocate).


Re-run the models. Post the results. More recent models presumably will do better, but firm knowledge about performance of the older models will give us useful information for the public policy debate. No matter what the results.

As the Romans might have said when faced with a problem like climate change: “Fiat scientia, ruat caelum.” (Let science be done though the heavens may fall.)

“In an age of spreading pseudoscience and anti-rationalism, it behooves those of us who
believe in the good of science and engineering to be above reproach whenever possible.“
— P. J. Roach, Computing in Science and Engineering, Sept-Oct 2004 — Gated.

Other posts in this series

These posts sum up my 330 posts about climate change.

  1. How we broke the climate change debates. Lessons learned for the future.
  2. A new response to climate change that can help the GOP win in 2016.
  3. The big step climate scientists can make to restart the climate change debate – & win.

For More Information

(a) Please like us on Facebook, follow us on Twitter, and post your comments — because we value your participation. For more information see The keys to understanding climate change and My posts about climate change. Also see these about models…

(b) I learned much, and got several of these quotes, from a 2014 presentations by Leonard A Smith (Prof of Statistics, LSE): the abridged version “The User Made Me Do It” and the full version “Distinguishing Uncertainty, Diversity and Insight“. Also see “Uncertainty in science and its role in climate policy“, Leonard A. Smith and Nicholas Stern, Phil Trans A, 31 October 2011.

(c)  Introductions to climate modeling

These provide an introduction to the subject, and a deeper review of this frontier in climate science.

Judith Curry (Prof Atmospheric Science, GA Inst Tech) reviews the literature about the uses and limitation of climate models…

  1. What can we learn from climate models?
  2. Philosophical reflections on climate model projections.
  3. Spinning the climate model – observation comparison — Part I.
  4. Spinning the climate model – observation comparison: Part II.

(d)  Selections from the large literature about validation of climate models

  • “How Well Do Coupled Models Simulate Today’s Climate?“, BAMS, March 2008 — Comparing models with the present, but defining “present” as the past (1979-1999).
  • “Should we believe model predictions of future climate change?”, Reto Knutti, Philosophical Transactions A, December 2008.
  • “Should we assess climate model predictions in light of severe tests?”, Joel Katzav, Eros, 7 June 2011.
  • “Reliability of multi-model and structurally different single-model ensembles“, Tokuta Yokohata et al, Climate Dynamics, August 2012. Uses the rank histogram approach.
  • “The Elusive Basis of Inferential Robustness“, James Justus, Philosophy of Science, December 2012. A creative look at a commonly given reason to trust GCMs.
  • “Test of a decadal climate forecast“, Myles R. Allen et al, Nature Geoscience, April 2013 — Gated. Test of one model’s forecasts over subsequent 10 years. Doesn’t state what emissions data  used for validation (scenario or actual). The forecast was significantly below consensus, and so quite accurate. Which is why we hear about it.
  • “Overestimated global warming over the past 20 years” by John C. Fyfe et al, Nature Climate Change, Sept 2013.
  • “Can we trust climate models?” J. C. Hargreaves and J. D. Annan, Wiley Interdisciplinary Reviews: Climate Change, July/August 2013.
  • “The Robustness of the Climate Modeling Paradigm“, Alexander Bakker, Ph.D. thesis, VU University (2015).
  • “Uncertainties, Plurality, and Robustness in Climate Research and Modeling: On the Reliability of Climate Prognoses“, Anna Leuschner, Journal for General Philosophy of Science, in press. Typical cheerleading; proof by bold assertion.

Like this:


September 24, 2015 in Modeling.

Pakistan on Monday came close to recording the highest successful run-chase in Test cricket courtesy their lower batting order. It would have been an apt honour for a team that defies cricketing nous.

At the start of the 144th over in Pakistan’s second innings, Yasir Shah had ambled through for a single to give the well-set Asad Shafiq the strike.

Pakistan were at 449/8 at the time in their chase of 490. The Pakistanis were expected to play out this over comfortably like they had the rest of the morning.

The two batsmen had shared a 70-run partnership for the ninth wicket and everything was going in their favour. They had not left too much to chance except for a slash or two.

They looked like they were batting in the first innings and building a lead.

Pakistan had failed to do anything of the sort in actuality in their first essay, when they had succumbed to 142 all out.

At one point in that innings, they had been at 67/8 in reply to Australia’s 429. Now, they were 449/8 and coolly pursuing their 490 target. However, within the next five balls, they lost the match.

Shafiq got a bouncer from Mitchell Starc that he was unable to fend. And, then, Yasir Shah, clouded by the idea of depending on Rahat Ali to see off the win, gave away his wicket.

Unpredictability thy name is Pakistani cricket

The adage of cricket being an unpredictable game becomes easier to mouth when one is talking about the Pakistani cricket team’s fortunes.

It is much easier to imagine Pakistan doing remarkable things in the bowling department than in batting. And this was, after all, against an Australian side that had been knocked out for 85 runs by South Africa only last month.

Yet, everything taking place today revolted against recent cricketing fortune and old cricketing history.

The highest ever run chase in a losing cause was New Zealand’s 451 against England in 2002. That had been achieved primarily via Nathan Astle’s blitzkrieg of 222 runs.

The New Zealand tail on that occasion had been there to just give the strike to Astle to do his thing, which was to pulverise the English bowlers everywhere.

They still fell short by 98 runs. It seemed like an effort done out of vengeful jest than controlled fury.

What Pakistan were up to in Brisbane was a studied demarcation of the run chase in front of them and they had 200 potential overs to achieve it.

So many chases are lost in the mind of the team that has to bat in the fourth innings because of the fear of not batting out a day or two.

Pakistan refreshingly looked at the number of overs available as an opportunity to do their thing, which was to win when no one gave them a fighting chance to do it.

The idea of winning a moral victory was easier to achieve than an outright one. Their resolve grew stronger when they were at 220/6.

Shafiq batted quite well in tandem with Mohammed Amir, Wahab Riaz and Yasir Shah to almost achieve the miraculous.

What India can learn

India might well have a proficient lower order in the form of Ravichandran Ashwin, Ravindra Jadeja and Jayant Yadav, who have rescued the team quite a few times this year.

But until they score more than double the runs of their top-order brethren, twice, away from home, in a day/night match, they have nothing much to brag about in comparison.

The need of this Indian team to do well abroad becomes more prominent in light of fairly sedate home series victories. None of the major cricketing nations do badly at home. The recent loss of Australia to South Africa only serves as an exception.

Another remarkable performance abroad this year was Pakistan’s 2-2 result against England.

They did not bow down after they had lost two matches in a row having won the first. India capitulated the last time they were in such a situation in England.

It might be easy to dismiss James Anderson’s recent mutterings against Virat Kohli’s batting as hypocritical, but the fact remains that Kohli and the rest of the India batsmen were clueless for most of that 2014 tour.

It is worth contemplating how they might play when they go to Australia and England in 2018.

The majority of the Indian batting order is likely to remain the same two years from now, but questions of experience and ability will linger.

The Indian spinning assets will be negated considerably then and India will need to find a functional pace line-up if they have any hope of doing well in these two countries.

As of now, injuries mark out the Indian pacers more than constant contributions. It is a bit dazzling to think of Ishant Sharma as a spearhead figure a la Zaheer Khan. But he has turned in a couple of good performances in England and Sri Lanka.

He and the rest of the fast bowlers will need to bowl consistently well in those swinging conditions, while the batsmen will have to weather the storm when it is their turn.

Pakistan, on the other hand, have a constant supply of fast bowlers who can do well with seam and swing available.

They might have struggled badly in the first Test against the Australian batsmen, but they certainly did the damage in England earlier this year. Who would bet against them doing the same in the remaining two Tests?

This article originally appeared on and has been reproduced with permission.


Leave a Reply

Your email address will not be published. Required fields are marked *