Weather or Not

Severe Outflow by R. Edwards

  • Home
  • About
  • Archives

Powered by Genesis

Human Weather Forecasting in an Automation Era, Part 3: Garbage In, Garbage Out

August 28, 2022 by tornado Leave a Comment

This short series (go to Part 1 or Part 2) arises from the recently published paper, “The Evolving Role of Humans in Weather Prediction and Communication“. Please read the paper first.

Objective verification of forecasts will remain hugely important, and the authors duly note that. But one factor not discussed (perhaps due to space limitations?) is the quality of the verification data. That matters…perhaps not to bureaucrats, who tend to overlook components of the verification sausage that provide context. But flawed verification datasets give you flawed verification numbers, even if the calculations are completely mathematically correct!

As someone who has analyzed and examined U.S. tornado, wind and hail data for most of my career, and published some research rooted in it, I can say two things with confidence:
1. It’s the most complete, precise and detailed data in the world, but
2. Precision is not necessarily accuracy. The data remain suffused with blobs of rottenness and grossly estimated or even completely fudged magnitudes, potentially giving misleading impressions on how “good” a forecast is.

How? Take the convective wind data, for example. More details can be found in my formally published paper on the subject, but suffice to say, it’s actually rather deeply contaminated, questionably accurate and surprisingly imprecise, and I’m amazed that it has generated as much useful research as it has. For example: trees and limbs can fall down in severe (50 kt, 58 mph by NWS definition) wind, subsevere wind, light wind, or no wind at all. Yet reports of downed trees and tree damage, when used to verify warnings, are bogused to severe numeric wind values by policy (as noted and cited in the paper). A patently unscientific and intellectually dishonest policy!

For another example, estimated winds tend to be overestimates, by a factor of about 1.25 in bulk, based on human wind-tunnel exposure (same paper). Yet four years after that research published, estimated gusts continue to be treated exactly like measured ones for verification (and now ML-informing) purposes. Why? Either estimated winds should be thrown out, or a pre-verification reduction factor applied to account for human overestimation. The secular increase in wind reports over the last few decades since the WSR-88D came online also should be normalized. That’s the far more scientifically justifiable approach than using the reports as-is, with no quality control nor temporal detrending.

For one more example, which we discussed just a little in the paper, all measured winds are treated the same, even though an increasing proportion come from non-AWOS, non-ASOS, non-mesonet instruments such as school and home weather stations. These are of questionable scientific validity in terms of proper exposure and calibration. The same can be said for storm-chaser and -spotter instrumentation, which may not be well-calibrated at a base level, and which may be either handheld at unknown height and exposure, or recording the slipstream if mounted on a vehicle.

Yet all those collectively populate the “severe” gust verification datasets also are used for training machine-learning algorithms — to the extent that actual, measured winds with scientific-grade, calibrated, verifiably properly sited instruments are a tiny minority of reports. With regard to wind reports, national outlooks, local warnings, and machine-learning training data use excess, non-severe wind data for verification, but because they all do, comparisons between them still may be useful, even if misleading.

Several of us severe-storms forecasters have noticed operationally that some ML-informed algorithms for generating calibrated wind probabilities put bull’s-eyes over CWAs and small parts of the country (mainly east) known to heavily use “trees down” to verify warnings, and that have much less actual severe thunderstorm wind (based on peer-reviewed studies of measured gusts, such as mine and this one by Bryan Smith) than the central and west. This has little to do with meteorology, and much to do with inconsistent and unscientific verification practices.

To improve the training data, the report-gathering and verification practices that inform it must improve, and/or the employers of the training data must apply objective filters. Will they?

This concludes the three-part series stimulated by Neil’s excellent paper. Great gratitude goes to Neil and his coauthors, and the handful who ever will read this far.

Filed Under: Weather Tagged With: data, data quality, education, forecast verification, forecasting, meteorology, numerical models, operational meteorology, quality control, science, severe storms, severe weather, storm observing, thunderstorm winds, understanding, verification, weather, wind, wind damage

“Acts of God” as Failures of People

August 19, 2011 by tornado Leave a Comment

The hits just keep coming…weather hits, that is, resulting in entirely preventable casualties and damage.

Last evening at the Omaha (OMA) airport, planes full of passengers sat out on the tarmac amidst two severe thunderstorms, both bearing hail and damaging wind, the second with extremely severe gusts measured at up to 92 mph at the airport. A CNN story indicates a Southwest plane was boarding while one of the severe storms was moving into the airport grounds.

How could this be? The airport was under severe thunderstorm warnings, had been in a severe thunderstorm watch since 1:30 p.m., and an upgrade to tornado watch at 6:40 p.m. before the second storm. As far back as shortly after midnight the night before, the OMA area was smack-dab in the middle of an enhanced probability specifically for damaging thunderstorm gusts. And yet those planes, crew and passengers got caught in exactly such storms.

Why?

Meteorologist Mike Smith has posted an illuminating and provocative BLOG entry about this event, with emphasis on the second storm.

As in the St. Louis airport-tornado incident of the past spring, that sort of unawareness of, and/or unwillingness to respond to, impending severe weather is absolutely inexcusable for any major public facility, much less an airport and airlines whose core business model and operations depend on…you guessed it…weather!

From abroad arises news of another five-death thunderstorm-wind event at a fairground, this time in Belgium. From Euronews:

Does that look familiar?

I’ll say again, these are not “Acts of God”, but instead, failures of people. Storms have roamed across the land for millennia. The difference now is that they are usually predictable. If we put in front of these storms such enticements to mayhem as airplanes full of passengers, casino or convention tents, flimsy stage rigs amidst thousands of unprotected outdoor spectators, or stick-like houses with little or no bracing that seem designed by Big Bad Wolf Architects LLC, then of course bad things will happen and people will suffer. The solution involves foresight, awareness and preparedness!

Filed Under: Weather Tagged With: airline safety, airport, airport preparedness, airport seafety, Belgium concert storm, concert, downdraft, fair, festivals, gust front, large venue, large-event venue, Omaha airport storm, outflow, severe storm, stadiums, state fair, Storm Ready, storm safety, thnunderstorm, thunderstorm safety, thunderstorm wind, thunderstorm winds, venue, venue preparedness, weather preparedness, weather safety

Preventable Large-Venue Tragedy in Indiana

August 14, 2011 by tornado 1 Comment

Going to the fair evokes fun memories for all of us who ever did: teen romance high atop Ferris wheels, the clank of wooden rings bouncing off bottles, the pops of balloons hit by darts, excited screams of kids on roller coasters high overhead, goats baying and cattle mooing in the livestock barns, the taste of a hot corny dog washed down by a cold soda, sticky fingers after cotton candy, persuasive demonstrations of miraculously effective vacuum cleaners and eyeglass defoggers, free samples in the food-and-fiber pavilion, registering to win a vacation to a remote wilderness lodge or tropical island, country and rock bands in concert across the way at the stage…

And now, for thousands of concert-goers at one fair, those memories now include the horrifying sight of fellow concert fans being crushed to death.

Last night, at the Indiana state fairgrounds, outflow winds from approaching thunderstorms toppled a stage rig onto folks gathering for a Sugarland show. Four people died, with over 40 more hurt (local media story…link may be temporary). [EDIT: CNN now reports five killed.] I am profoundly saddened that this happened for many reasons, one of which is because it didn’t need to.

I’ve deliberated whether to show the following video, out of respect for the dead and their families. Yet a little voice inside says, “they would want this shown if it could prevent more folks from suffering the same end”. So here you go (language alert):

If you are a venue manager or event promoter, click “play” again and consider the possibility that nature will call your facility’s name next–and that those people being crushed to death down there are your paying customers, crews, or family and friends. Sobering, I hope…and also, I hope, fodder for action in the form of developing a comprehensive and publicly known preparedness plan that can be kicked into action immediately at all time scales leading up to each and every event.

This clock has been ticking for a long time. It alarmed last night in a deeply troubling way. And it still ticks.

Large-venue weather disasters are not “acts of God”, they are failures of people. Why? Because the great majority of time, such weather now is predictable. I know this because the great majority of time in the modern era of forecasting, the potential for severe weather in the area is predicted! Yesterday, Indianapolis was in a severe thunderstorm outlook, watch and warning. And yet, the show must go on…really?

The problem is nothing new; as Les Lemon and I noted in nine years ago, covering decades of threat. In presentations nationwide, we have called attention to this matter using dozens of examples. In several expositions in this space, I’ve written about assorted “near disasters” (in Minnesota, Oklahoma, Texas, and Georgia) and the idea of “Atmospheric Terrorism” as a conceptual starting point for motivating and undertaking preparedness. Behind the scenes, we work to educate media and venue organizations, and mostly get friendly and open reception in doing so. Les has been an absolute bulldog about this in terms of gathering people together who could make a difference, and I am sure his efforts and those of others eventually will save lives.

What else? This is damn frustrating, because events like the Indy stage collapse are so preventable. In perusing the website for the fair as of this writing, I find nothing even remotely resembling a severe-weather plan of action…not even any severe-storm shelters marked on the fairgrounds map. Alas, this missing-information phenomenon is nothing new either, in my experience of searching venue websites immediately after they’ve experienced a disaster or nearly one.

Sometimes it seems as if folks like Les and I, along with Greg Blumberg (a super-sharp OU student who has studied amusement-park readiness), Al Moller (ret. NWS), and others in meteorology and venue preparedness, have been pushing a boulder uphill with the futility of Sisyphus. Awareness and action propagate with glacial pace; meanwhile event after event gets smacked by severe weather amidst seeming disorganization and non-readiness masquerading as “plans”.

Now these incidents are getting deadly. How much more of this must happen? Whether you are an organizer or customer, how prepared is the venue where your next concert, fair, festival or race will be held? What is your own personal, individual severe-storm safety plan for the next such event you’ll attend?

Filed Under: Weather Tagged With: concert, downdraft, fair, festivals, gust front, large venue. large-eventvenue, outflow, severe storm, stadiums, state fair, Storm Ready, storm safety, thnunderstorm, thunderstorm safety, thunderstorm wind, thunderstorm winds, venue, venue preparedness, weather preparedness

Search

Recent Posts

  • Scattershooting 230128
  • A Thanksgiving Message
  • Human Weather Forecasting in an Automation Era, Part 3: Garbage In, Garbage Out
  • Human Weather Forecasting in an Automation Era, Part 2: Lessons of Air France 447
  • Human Weather Forecasting in an Automation Era, Part 1: Situational Understanding

Categories

  • Not weather
  • Photographic Adventures
  • Scattershooting
  • Weather
  • Weather AND Not
@SkyPixWeather

- January 30, 2023, 3:38 am

@cschultzwx @TwisterKidMedia So many holds don’t get called. That looked quite familiar. I know this as a Cowboys/Micah Parsons fan.
h J R
@SkyPixWeather

- January 30, 2023, 3:27 am

@TwisterKidMedia @sdantwx Worst officiating I’ve ever seen was in a college game too, and Andrew should know this one. https://t.co/pC5fTkFFrF
h J R
@SkyPixWeather

- January 30, 2023, 3:20 am

@TwisterKidMedia @tempestchasing I still don’t understand fully WTH happened w/the “unheard whistle” clock debacle and play that wasn’t. That was bizarre.
h J R

Blogroll

  • CanadianTexan
  • Chuck's Chatter
  • Cliff Mass Weather & Climate
  • Digital Photography Review
  • DMN Dallas Cowboys BLOG
  • Dr. Cook's Blog
  • Dr. JimmyC
  • E-journal of Severe Storms Meteorology
  • Eloquent Science
  • Image of the Week
  • Jack's Cam Wall
  • Jim LaDue View
  • Laura Ingraham
  • MADWEATHER
  • Michelle Malkin
  • Photography Attorney
  • Severe Weather Notes
  • SkyPix by Roger Edwards
  • Tornatrix
  • With All My Mind

Meta

  • Log in
  • Entries feed
  • Comments feed
  • WordPress.org