Blog - June 2008

Brits Rely on Rumsfeld in Climate Article

Back in 2006, in a paper describing an update to one of the standard global surface temperature data sets, HadCRIT3, Brohan et al. (2006, JGR, doi:10.1029/2005JD006548) included the following passage:

To use the data for quantitative, statistical analysis, for instance, a detailed comparison with GCM results, the uncertainties of the gridded anomalies are a useful additional field. A definitive assessment of uncertainties is impossible, because it is always possible that some unknown error has contaminated the data, and no quantitative allowance can be made for such unknowns. There are, however, several known limitations in the data, and estimates of the likely effects of these limitations can be made (Defense secretary Rumsfeld press conference, June 6, Back to disarmament documentation, June 2002, London, The Acronym Institute (available at www.acronym.org.uk/docs/0206/doc04.htm)). This means that uncertainty estimates need to be accompanied by an error model: a precise description of what uncertainties are being estimated. 

Here's what Rumsfeld said:

Now what is the message there? The message is that there are no "knowns." There are things we know that we know. There are known unknowns. That is to say there are things that we now know we don't know. But there are also unknown unknowns. There are things we don't know we don't know. So when we do the best we can and we pull all this information together, and we then say well that's basically what we see as the situation, that is really only the known knowns and the known unknowns. And each year, we discover a few more of those unknown unknowns. It sounds like a riddle. It isn't a riddle. It is a very serious, important matter.

The analogy that Brohan et al. were making to science is that:

1. data has some specific value (the known knowns);
2. data always has some uncertainty associated with it;
3. some of those uncertainties we understand enough to make quantitative uncertainty estimates (the known unknowns);
4. some of those uncertainties we don't even know well enough to estimate their size (the unknown unknowns).

Brohan et al. had three known unknowns for the sea surface temperature record: the random error of an individual measurement (this goes down as more measurements are taken), the sampling error of incomplete data coverage in a grid box or over the globe (this goes down with more measurements too), and the bias correction error, which is independent of number of measurements.  The bias being corrected was the switch from lowering a bucket over the side of a ship, bringing up some water, and measuring its temperature (the old way), and measuring the temperature of the water being directly drawn into the ship to cool the engine (the new way).  Since we don't know precisely the difference caused by the measuring techniques, there's some residual error left after bias correction.

These three known unknowns were estinated to lead to a 95% uncertainty range of about +/- 0.12 C back at the beginning of the 20th century, +/- 0.1 C in more recent times.  These are the uncertainty ranges normally quoted.  But this is not the actual uncertainty, because there're also the unknown unknowns.

The New Stuff 

A paper being published tomorrow in Nature Geoscience (Thompson et al. 2008 NG, doi:10.1038/nature06982) has identified a major unknown unknown.  I'll chat with David Thompson in a couple of weeks to get the full story.  In the paper, they show that once you remove the ENSO (El Nino) variability from the global mean sea surface temperatures, you're left with a sudden, unexplained drop in temperature of almost half a degree C around 1945.  This drop doesn't show up at all in the land surface data, making it suspicious. 

With the recent digitization and online availability of information about different types of measurements over time, it turns out that starting abruptly after World War II and for a few decades thereafter, a significant fraction of the measurements were by bucket rather than by water intake.  This caused an uncorrected cool bias: as the water is brought up to the deck of the ship, it literally cools.  The expected magnitude of this bias, about 0.3 C overall, is consistent with the unexplained drop in global sea surface temperatures.  Note that the 0.3 C bias between 1945 and 1948 is about three times the 95% uncertainty range.

As the paper points out, this doesn't change the century-long trend, because it's an error in the middle part of the record, but it does affect the details of the mid-century record.  

To me, the importance of the paper is in reminding us that there are unknown unknowns.  The global temperature record is one such case.  The accuracy of the century-long trend is estimated at 0.1 C to 0.2 C, but that just includes the known unknowns, and the true uncertainty is unquantifiable but larger.  Another such case is sea level rise, where the IPCC projections only include the quantifiable parts from thermal expansion and glacier melting, and the known but hugely unknown contribution from Greenland and West Antarctic ice sheets is not included.  

At least with sea level rise, we know the sign of the effect from the part left out.  With global temperatures, the remaining unknown unknowns could be of either sign.  I'm glad recent decades have brought multiple observing systems with different ways of detecting climate change, because reliance on one data source is inherently dangerous.