Tornado watchers are missing more storms, giving shorter notice – and saving more lives?

On April 27, 2011, TV weatherman James Spann warned viewers that a “large, multiple-vortex tornado” was bearing down on Tuscaloosa, Ala.

“You should have been in your safe place 20 minutes ago,” he said as a camera tracked the funnel clouds, “but if by chance you’re hearing me at the last minute on the radio, get into a safe place right now!”

Despite these warnings, 252 Alabamians died that day, victims of the fourth-deadliest tornado season in US history. Mr. Spann, who still serves as chief meteorologist for Birmingham’s WBMA-TV, drew a clear lesson from the tragedy.

“What we learned that day is that physical science could not have been better,” he remembers in a phone interview with The Christian Science Monitor, “but what maybe we don't understand is the social science part of it.”

The nature of twisters makes their exact timing and location difficult to predict. But since 2011, sociological research and storms like Alabama’s have spurred a shift in forecasting practices. Meteorologists now aim to reduce false alarms – even at the risk of missed storms and delayed warnings – in the hopes that residents will heed the warnings they do issue.

“There is a recognition that the false alarm rate is something that we need to take into consideration,” explains economist Kevin Simmons, who researches the statistics of natural disasters.

A high-stakes trade-off

Tornado forecasters’ performance gets measured by three key numbers:

• False-alarm rate (FAR), the percentage of tornado warnings issued for storms that never touch down
• Probability-of-detection (POD), the percentage of warnings that accurately predict tornadoes
• “Lead-time,” the minutes between when a warning is issued and when the storm touches down

“Since around 2011, both tornado lead-time and detection have gotten worse. But the false alarm rate has improved (decreased),” The Washington Post reported on April 20.

Harold Brooks, a senior scientist at the National Severe Storms Laboratory, explains that he and other forecasters face a trade-off: Longer lead-times and higher POD’s versus a lower false alarm rate. As forecasters wait for more evidence – from radar, spotter teams, and other sources – before sounding the alarm, they issue fewer false ones. But they’re also more likely to miss some, and give residents less time to take cover.

“Our skill hasn’t changed” in recent years, he tells the Monitor over the phone, but they have employed “a higher threshold for warning.”

Despite the obvious risks of missing a storm or cutting down on warning time, Dr. Simmons, a professor of economics at Austin College, in Sherman, Texas, says there’s merit in reducing false alarms. “If a tornado strikes an area with a higher-than-average false alarm rate, it's more likely that that tornado would generate fatalities [than in an area] with lower-than-average false alarm rates,” he tells the Monitor.

In 2009, he and economist Dan Sutter, currently at Alabama's Troy University, assessed the trade-off between false-alarm rates and POD, finding “strong [statistical] evidence that a higher local, recent FAR significantly increases tornado fatalities and injuries.” As residents hear one false alarm after another, the “cry-wolf effect” takes hold, and they’re less disposed to heed the one warning that could save their lives.

Two years after they published these findings, the devastating 2011 season bore them out. In Birmingham, where almost 80 percent of warnings were false, Spann insists, “there's no doubt in my mind a high false alarm ratio in 2011 killed people."

The National Weather Service (NWS) reached a similar conclusion in Joplin, Mo., which suffered one of that year’s worst twisters. It found that “the perceived frequency of siren activation (false alarms) led a large number of [residents] to become desensitized or complacent to this method of warning.” Dr. Brooks cites this survey as a key reason behind the new focus on false-alarm rates. It's since edged down, from 73 percent in 2011 to 70 percent in 2015.

Sure enough, the same period saw forecasters detecting fewer storms, with POD going from 75 percent to 58 percent. But Eric Waage, director of emergency management for Hennepin County, Minn., says not all of the missed storms give cause for concern.

“The most problematic thing we have are these small ... EF-0 tornadoes,” he tells the Monitor via phone. These are the weakest storms on the Enhanced Fujita scale that measures tornado strength in the United States and Canada based on the damage they cause. Spann explains that, because of their short duration, “trying to warn for those suckers ... is like playing Whack-a-Mole."

These days, forecasters may find it harder to get enough evidence to warn against these storms. But they also have more room for error. EF-0 and EF-1 storms occasionally make the NWS’s killer tornado list, but most deadly storms are EF-2 and up. Mr. Waage says that “in most places in our state, those small ones aren't really a huge problem. They're hitting cornfields and forests.”

The third key tornado metric, lead-time, has also dropped amid the NWS's focus on false alarms, from 15 minutes in 2011 to just eight in 2015. But that national trend obscures local variations. Spann remembers that April 27, 2011 saw lead-times as high as 40 minutes. Since that tragic day, the local NWS office's average lead-times have dropped slightly – and false alarms have plunged.  

“We've got to get the FAR down,” Spann says, “and if we lose a little lead-time, I don't have any problem with that.”

'We are better than that'

A 2011-caliber season hasn’t yet tested this new mind-set, and Brooks, speaking with the Post’s Jason Samenow, cautioned that social science research on this topic is “difficult to conduct and often inconclusive.”

But it could provide lifesaving guidance for the Midwest’s “Tornado Alley” and the Southeast’s “Dixie Alley,” at least until the next major leap in meteorology.

Last week, President Trump signed a bill aiming to extend tornado prediction time beyond one hour. But Brooks cautions that, to lengthen lead-times without raising too many more false alarms, “we need to move a long way from that [current] skill line, and that's hard.”

He says it’s happened before, thanks to Doppler radar and other innovations in the 1990s. One government program, Warn on Forecast, aims to repeat the feat with numerical models, but practical applications remain “years away.”

For the moment, forecasters are fine-tuning the balance of lead-time, POD, and false-alarm rate, and authorities are trying to get more people to respond to warnings. “The missing link still is that public awareness,” says Hennepin County’s Waage. To minimize confusion when the skies darken, for instance, he explains that Minnesota has recently standardized its siren procedures.

Birmingham, Ala., has also seen a push for greater tornado awareness since 2011, and Spann thinks it’s heading in the right direction.

“There have been too many funerals on my watch in 38 years,” as a meteorologist, he reflects. “And we are better than that. And now, working with the social science people, I think that's going to make a huge difference.”

[Editor's note: This article has been updated to correctly state the university where Dan Sutter currently works]