Back in August, Czech Physicist Luboš Motl posted a blog that discussed the IPCC’s 95 percent “certainty” that global warming is mostly man-made and the subsequent runaway usage of that figure from the media to imply large amounts of money should immediately be spent to stop global warming. Motl, a highly-trained physicist, then contrasted that with how “95% certain” findings are more sensibly applied in particle physics.
I just came across his post now, and was struck most to learn how, in the world of particle physics, a 95% confidence level is “incredibly low,” Motl writes:
As I said, the 95% confidence level is known as the 2-sigma confidence in hard sciences such as particle physics. Particle physics experiments have brought us hundreds of 2-sigma excesses – and lots of much larger (more confident) excesses – and a vast majority of them turned out to be flukes. When more data were accumulated, these excesses just went away. Such things inevitably occur all the time.
Because people keep on looking for new effects, they inevitably encounter flukes that look like a new effect but the effect actually doesn’t exist. A priori, a new effect is always a pretty unlikely thing so if you look at the history of particle physics, most of the 2-sigma deviations were really flukes – results of coincidences that shouldn’t have been paid any attention to.
One important distinction Motl identifies before making this analogy is that 95% confidence levels in particle physics are calculated differently than how the IPCC calculates their confidence level, namely, the IPCC does no calculating at all, in fact they come up with their percentage level through a voting process.
Of course, failure to calculate a credible percentage figure is not entirely the IPCC’s fault, the enormous complexity of climate science stands in the way of allowing any person or group of persons to simply collect concrete data and then plug them into a clearly-defined calculation to be done by a computer. So they have a pretty good excuse one could say, but it’s still just an excuse, and science doesn’t recognize excuses, even if politics might.
(Photo credit: Oregon State University)