If a tree falls in the forest: Overcoming challenges in U.S. severe convective storm observations

Author: Tom Sabbatelli, Director - Product Management, Moody's

For carriers insuring severe convective storm (SCS) perils—tornadoes, hail, and damaging winds in the United States, the last few years have been frenetic—and costly. 

In each of the last three years, reported U.S. SCS insured losses have exceeded US$50 billion, eclipsing even hurricane losses. Unlike hurricane losses dominated by ‘megaevents’ or named storms, small to medium-sized SCS events typically occur throughout the year, aggregating into significant total annual losses. As these smaller, more frequent SCS events accumulate over the year, they also add to claims handling expenses, materials, and labor costs.

It has become clear that the insurance market requires a model that can change how we quantify SCS risk. We developed the new Moody’s RMS™ North America Severe Convective Storm HD Models with this market need at the forefront of our minds, and starting from the ground up.

As with any new model or modeling approach, the output needs to be validated before an insurer can be confident in its results. But with an SCS model, validation itself can result in challenges.

Given the inherent biases in observation data that only span a relatively short period and the uncertainty in SCS observations, validating modeled SCS hazard is not a straightforward task. Let’s look at where these inherent biases originate from.

Observation limitations

Observations from the Storm Prediction Center (SPC), a division of NOAA’s National Weather Service, form the backbone of severe weather climatology in the United States. Since the 1950s, the SPC has maintained an archive of severe storm reports, each documenting the time, location, and severity of tornadoes, hailstorms, and wind events. 

Figure 1: Storm Prediction Center Storm Reports from May 8, 2024 (Source: NOAA SPC)

It is tempting to take SPC storm reports at face value and draw trends in the behavior of sub-perils or intense storms. But in catastrophe modeling, when working with only decades of data to model tens of thousands of years of activity, developers must be cautious not to overfit to uncertain ‘trends.’

These SPC observations are just that, an observation; someone has seen a tornado or hailstorm and reported it. From this reporting approach, several SPC data trends can arise due to reporting bias. It calls to mind that age-old saying: "If a tree falls in a forest and no one hears it, does it make a sound?"

There may be systematic underestimation. intense storms in urban areas may receive many reports from ‘storm watchers’, but storms in less-populated, rural areas, especially small-to-medium-sized events, may be less reported, skewing the true frequency and intensity of storms.

Thanks to our SCS modelers, Moody’s takes great pains to correct biases in the historical record, but a (re)insurer may have limited resources to take these corrections into account for its validation projects. So, how should a user begin to validate an SCS hazard model?

Model validation toolkit: How to overcome observational uncertainty

Here are three opening thoughts to guide model validation efforts, with an emphasis on where we know observations to be most reliable:

1. Focus on data from populated regions

We can be most confident in reports from suburban and urban areas, where population density and infrastructure ensure that severe weather events are well observed and their impacts accurately recorded.

Modern technology and exposure growth also give us the highest confidence in more recent parts of the record, with more detection systems and people available to observe storms.

To overcome reporting bias, Moody’s trains its machine learning techniques to correlate reports from densely-populated regions with the convective conditions at the time, as defined by weather reanalysis data. 

We can combine these techniques with expected levels of undercounting from across the entire country to fill in the observational gaps.

As a result, a model should generally produce more severe hazard than the observed record, especially in rural areas and in earlier years, while remaining consistent with observed severity in urban and suburban regions. 

Figure 2: Comparison of Moody’s modeled (blue) and SPC observed (green) annual tornado counts in U.S. cities (left) and for the entire U.S. (right) (Source: Moody’s)

2. Focus on data from the spring

Let’s follow this thread to discuss a nuance in SCS seasonality that might not be top-of-mind. Convective conditions ripe for tornado development are more prevalent in the southeast U.S. during the spring and then shift to the northern Plains in the summer.

But with a lower population density in the Plains, we must consider that the record likely undercounts more in the summer than in the spring.

Figure 3: Average tornado risk in March and June, showcasing differences in event seasonality (Source: The Weather Channel)

Therefore, if seasonality is considered in a validation analysis, it may be reasonable to expect a model to exhibit a closer fit to SPC observations in the spring than in the summer.

3. Focus on daily and seasonal statistics

Although the total number of tornado, hail, and wind events may be undercounted, the existing record still provides key benchmarks on sub-peril correlation. Even just one tornado, hail, or wind report in a day gives us critical insight into event seasonality.

A model should replicate the overall temporal distribution of sub-peril occurrence. This extends to sub-peril combinations: days containing all three sub-perils tend to peak in the summer, while days with just hail and wind peak in early Fall.

Figure 4: Smoothed distribution of historical daily combinations of hail (h), tornado (t), and straight-line wind (w) in North America, by month (Source: Moody’s)

Moody’s also gathers a library of additional data sources, including radar, station data, and even insurance claims, to provide a more complete picture of storm activity.

(Re)insurers will often also have access to similar data, and we encourage its use for a thorough model evaluation.

Moody’s is all set to change the landscape of SCS modeling

We will soon release a new high-definition (HD) severe convective storm model for the U.S. market—a model featuring several innovations.

Few peril models benefit more from recent advancements in cloud computing, leveraged by Moody’s Intelligent Risk Platform™, than this HD model. This enhanced computational power, along with the latest scientific knowledge, unlocks modeling of a vast number of realistic stochastic severe convective storm events—ranging in size from a large, multi-state, multi-day outbreak to a smaller, localized event—all at a high-resolution basis.

The hazard of each event is built on an innovative, physically based approach designed to represent the true variability in space and time of hail, tornado, and straight-line wind, across both significant and minor events.

The conversation on SCS modeling often stops at hazard, but a model must match hazard with equally skillful vulnerability and financial modeling.

Our HD model will introduce a broader range of damage functions that extend to renewable energy, calibrated using over US$55 billion in location- and policy-level claims provided by our insurance carrier partners—the largest such library amassed for a model in Moody’s history.

And, our HD financial model explicitly captures the probability of no claim and complete loss, a critical enhancement for low-severity (hail) and high-severity (tornado) hazards.

These enhancements will not only provide a more comprehensive understanding of loss severity but also improve the modeling of each sub-peril to more accurately reflect today’s loss occurrences, for more informed decisions on pricing, capital allocation, and exposure management for this frequent peril.

For more information about the upcoming model, please contact Steve Drews, Moody’s Model and Product Specialist for Severe Convective Storms and Winter Storms, here. We look forward to continuing to support your risk management needs with our market-leading solutions.