How Accurate are Weather Forecasts?

In Canada’s 2013 federal budget, finance minister Jim Flaherty announced that an additional $248 million would be spent over five years to strengthen Canada’s meteorological services, a part of Environment Canada. Around the world, billions are spent every year on weather forecasting and the weather channel in North America is a huge hit with viewers and advertisers. Apparently, watching floods, fire, tornadoes, hurricanes and other disasters is profitable and fun (as long as it’s happening somewhere else).

But is all that investment of money and time worth it? Is it any better than an old guy standing on a hill and telling you, “Yep, it’s gonna rain”?

Maybe not. 

Overall success rates for weather forecasts generally settle at 77% so it’s better than flipping a coin. Accuracy in Palm Beach, Florida is higher than for a northern shelf city like Ottawa where there is much more variability in climate but the differences are small, perhaps 75% versus 79%. So far, so good.

But the real issue is not whether the Meteorological Service of Canada can accurately predict the high temp tomorrow (within three degrees, the generally accepted range considered a success) or whether it will rain or snow or hail or flood… The real issue is can they predict when a change will occur.

Let’s say you are a weather forecaster for Honolulu and a high pressure systems settles in for the next 90 days producing average highs of 31, 32 and 32 degrees Celsius every June/July/August and lows of 23, 24 and 24 respectively. To say that they have 90 successes out of 90 seems wrong. All 90 such days are essentially the same so they have one success assuming they accurately predict when that system arrives failing which they have zero right.

So it is much more interesting to measure how a weather forecaster is doing in terms of predicting weather changes not weather itself. This is certain to be much less than the 77% number you see many such organizations quoting. And, by the way, none of them seem to report accuracy this way. It would be too discouraging. 

On the freakonomics blog, JD Eggleston, in a study of local weather for Kansas City, found that a forecaster who always and arbitrarily predicted that it would never rain would be right 86.3% of the time. Three of KC’s local TV channels had 1-day out success rates above this– they were right 87% of the time. The difference is unlikely to be statistically significant.

So if you are wondering whether weather forecasts are accurate and dependable, well, you have a right to question that.

@profBruce

@quantum_entity

Postscript: I’ve been thinking about this subject for quite a while. I included a discussion about weather in book 1 of Quantum Entity trilogy. In this scene, Damien Bell has taken Ellen Brooks out for a sail in an Albacore on the Charles River in Boston. Damien is having fun; Ellen, her first time in a sailboat, is not:

“Hey Ellen, ever heard the expression, ‘Pink at night, sailor’s delight’?” Damien asks her with a twinkly smile as he comes back inboard.

“Nope. I suppose it’s some kind of rude joke about women that men seem to like to tell,” comments a completely unimpressed Ellen.

“Nothing of the kind. The full refrain is, ‘Pink at night, sailor’s delight. Pink in the morning, sailors take warning.’ It’s a sailor’s way of predicting clear or stormy weather. Just look for pink colours either at sunset or sunrise, and you’ll get an idea about what’s ahead. It’s a pretty useful rule of thumb and can even help young, female executives decide what they’re gonna wear that day or the next day—like whether to take an umbrella, a sweater, or a bikini with you.” Unbidden, he suddenly finds himself wondering what Ellen would look like in a bikini. Nice, he’s sure.

“What was the sunset like last night?” she asks.

“Exactamundo! From now on, maybe you’ll notice sunrises and sunsets instead of working all the time, Ms. Brooks. Look up, see the world, and experience it in real time, Ellen.”

“So what was it?” Ellen asks again.

“It was quite pinkish!”

“Delighted to hear that.”

“Hah. This’ll be like when you learn a new word or expression for the first time, and you wonder how you lived a few decades without ever hearing it before; and then, for the next few weeks, you’ll hear it everywhere.”

“Ah, I’m not planning on changing careers any time soon, Dr. Bell,” Ellen says. “Meteorology seems like a total waste of time to me anyway. What kind of industry produces results that, after trillions of New Dollars invested in gear, is wrong more often than an old geezer standing on a ridge, overlooking his farm at sunrise and saying, ‘Yep, it’s gonna rain!’?”

Damien laughs. “You just proved my point, QED. It’s as nice a day as you could wish for… as predicted.” Ellen is seeing another side to him—when he’s out on the water, he seems much freer, happy even. He’s also become like a part of this boat.