Weather or Not

Severe Outflow by R. Edwards

  • Home
  • About
  • Archives

Powered by Genesis

Human Weather Forecasting in an Automation Era, Part 3: Garbage In, Garbage Out

August 28, 2022 by tornado Leave a Comment

This short series (go to Part 1 or Part 2) arises from the recently published paper, “The Evolving Role of Humans in Weather Prediction and Communication“. Please read the paper first.

Objective verification of forecasts will remain hugely important, and the authors duly note that. But one factor not discussed (perhaps due to space limitations?) is the quality of the verification data. That matters…perhaps not to bureaucrats, who tend to overlook components of the verification sausage that provide context. But flawed verification datasets give you flawed verification numbers, even if the calculations are completely mathematically correct!

As someone who has analyzed and examined U.S. tornado, wind and hail data for most of my career, and published some research rooted in it, I can say two things with confidence:
1. It’s the most complete, precise and detailed data in the world, but
2. Precision is not necessarily accuracy. The data remain suffused with blobs of rottenness and grossly estimated or even completely fudged magnitudes, potentially giving misleading impressions on how “good” a forecast is.

How? Take the convective wind data, for example. More details can be found in my formally published paper on the subject, but suffice to say, it’s actually rather deeply contaminated, questionably accurate and surprisingly imprecise, and I’m amazed that it has generated as much useful research as it has. For example: trees and limbs can fall down in severe (50 kt, 58 mph by NWS definition) wind, subsevere wind, light wind, or no wind at all. Yet reports of downed trees and tree damage, when used to verify warnings, are bogused to severe numeric wind values by policy (as noted and cited in the paper). A patently unscientific and intellectually dishonest policy!

For another example, estimated winds tend to be overestimates, by a factor of about 1.25 in bulk, based on human wind-tunnel exposure (same paper). Yet four years after that research published, estimated gusts continue to be treated exactly like measured ones for verification (and now ML-informing) purposes. Why? Either estimated winds should be thrown out, or a pre-verification reduction factor applied to account for human overestimation. The secular increase in wind reports over the last few decades since the WSR-88D came online also should be normalized. That’s the far more scientifically justifiable approach than using the reports as-is, with no quality control nor temporal detrending.

For one more example, which we discussed just a little in the paper, all measured winds are treated the same, even though an increasing proportion come from non-AWOS, non-ASOS, non-mesonet instruments such as school and home weather stations. These are of questionable scientific validity in terms of proper exposure and calibration. The same can be said for storm-chaser and -spotter instrumentation, which may not be well-calibrated at a base level, and which may be either handheld at unknown height and exposure, or recording the slipstream if mounted on a vehicle.

Yet all those collectively populate the “severe” gust verification datasets also are used for training machine-learning algorithms — to the extent that actual, measured winds with scientific-grade, calibrated, verifiably properly sited instruments are a tiny minority of reports. With regard to wind reports, national outlooks, local warnings, and machine-learning training data use excess, non-severe wind data for verification, but because they all do, comparisons between them still may be useful, even if misleading.

Several of us severe-storms forecasters have noticed operationally that some ML-informed algorithms for generating calibrated wind probabilities put bull’s-eyes over CWAs and small parts of the country (mainly east) known to heavily use “trees down” to verify warnings, and that have much less actual severe thunderstorm wind (based on peer-reviewed studies of measured gusts, such as mine and this one by Bryan Smith) than the central and west. This has little to do with meteorology, and much to do with inconsistent and unscientific verification practices.

To improve the training data, the report-gathering and verification practices that inform it must improve, and/or the employers of the training data must apply objective filters. Will they?

This concludes the three-part series stimulated by Neil’s excellent paper. Great gratitude goes to Neil and his coauthors, and the handful who ever will read this far.

Filed Under: Weather Tagged With: data, data quality, education, forecast verification, forecasting, meteorology, numerical models, operational meteorology, quality control, science, severe storms, severe weather, storm observing, thunderstorm winds, understanding, verification, weather, wind, wind damage

Maria’s Destruction of San Juan Radar

September 25, 2017 by tornado Leave a Comment

Recall that when Hurricane Maria moved into Puerto Rico, the San Juan WSR-88D (Doppler radar for my non-weather friends) failed in the outer eyewall. Here’s what resulted:

The dome and antenna blew away to who-knows-where, and the steel framing itself appears slightly torqued. Trees all around are shredded to varying degrees, as should be expected with eyewall winds of an upper-echelon hurricane. The mowed-down trees in the foreground point uphill toward the radar, telling us the wind direction at the time they fell, and part of the source for embedded debris. The image below represents the last 0.5-degree reflectivity data acquired and transmitted before the radar was lost.

Now, thanks to a series of photos from NWS San Juan, two of which I include here, we can confirm why, and apply knowledge gained from damage surveys of other structures (and the Miami radar in Hurricane Andrew) to this weather-sensing structure. That is quite unfortunately ironic.

From available velocity data prior to failure (example below), we know that the radar — which sits high on a hill exposed to stronger flow than lower elevations receive — was hammered by sustained winds over 100 mph and gusts at that elevation near 150 mph. Those winds certainly contained leaves, sticks, and perhaps branches torn from nearby trees. The resulting debris-peppered airstream battered the dome piece by piece and gust by gust, until panels started loosening and coming apart under the combined stress of wind forces and high-velocity “sandblasting” effects.

Once the first panels started prying apart, the flow got inside and the entire dome shredded, exposing the antenna fully to forces for which it wasn’t designed. Being an airfoil, the dish tore away from the base easily and blew away. The entire process of destruction probably took only a few seconds, at most, from initial breach of the panels. To my knowledge, a WSR-88D hasn’t been struck directly by a significant (EF2+) tornado; however, given the windspeeds likely involved in this event, and the wind-resistance specifications of the radar structures (below, source link) one should expect similar results.

A speed of 60 m/s is the same as 134 mph. Of course the failure alone doesn’t prove the windspeed. We don’t know if the radar was built to those specs, nor to what extent the near-certain presence of flying debris played a role. Regardless, can the radar structures be engineered retroactively and cost-effectively against stronger flow?

Of course, the staff at San Juan is dealing with far more pressing matters personally and logistically than missing radar pictures and mechanical engineering of radomes. Still, it won’t help them in their public-warning mission to be without this crucial tool for at least some number of months, depending on how long it takes parts can be scavenged from elsewhere for reconstruction purposes, then shipped in and reconstructed, amidst all the island’s other damage.

Filed Under: Weather Tagged With: damage, hurricane, Hurricane Maria, hurricanes, Puerto Rico, radar, radars, radome, reflectivity, San Juan, storms, velocity, weather, weather damage, wind damage, wind engineering, WSR-88D

Search

Recent Posts

  • Scattershooting 230128
  • A Thanksgiving Message
  • Human Weather Forecasting in an Automation Era, Part 3: Garbage In, Garbage Out
  • Human Weather Forecasting in an Automation Era, Part 2: Lessons of Air France 447
  • Human Weather Forecasting in an Automation Era, Part 1: Situational Understanding

Categories

  • Not weather
  • Photographic Adventures
  • Scattershooting
  • Weather
  • Weather AND Not
@SkyPixWeather

- January 30, 2023, 3:38 am

@cschultzwx @TwisterKidMedia So many holds don’t get called. That looked quite familiar. I know this as a Cowboys/Micah Parsons fan.
h J R
@SkyPixWeather

- January 30, 2023, 3:27 am

@TwisterKidMedia @sdantwx Worst officiating I’ve ever seen was in a college game too, and Andrew should know this one. https://t.co/pC5fTkFFrF
h J R
@SkyPixWeather

- January 30, 2023, 3:20 am

@TwisterKidMedia @tempestchasing I still don’t understand fully WTH happened w/the “unheard whistle” clock debacle and play that wasn’t. That was bizarre.
h J R

Blogroll

  • CanadianTexan
  • Chuck's Chatter
  • Cliff Mass Weather & Climate
  • Digital Photography Review
  • DMN Dallas Cowboys BLOG
  • Dr. Cook's Blog
  • Dr. JimmyC
  • E-journal of Severe Storms Meteorology
  • Eloquent Science
  • Image of the Week
  • Jack's Cam Wall
  • Jim LaDue View
  • Laura Ingraham
  • MADWEATHER
  • Michelle Malkin
  • Photography Attorney
  • Severe Weather Notes
  • SkyPix by Roger Edwards
  • Tornatrix
  • With All My Mind

Meta

  • Log in
  • Entries feed
  • Comments feed
  • WordPress.org