Weather or Not

Severe Outflow by R. Edwards

Human Weather Forecasting in an Automation Era, Part 1: Situational Understanding

by Leave a Comment

This short series arises from the recently published paper, “The Evolving Role of Humans in Weather Prediction and Communication“. Please read that paper first, before proceeding.

Neil Stuart always has been a thought-provoking author and discussion partner at conferences, regarding the human role in forecasting—most certainly including communication of forecasts. He lead-authored this new article in Bulletin of the AMS that fortunately is not paywalled, and shouldn’t be, given its importance. [I posit that no published atmospheric science should be paywalled, but that’s another issue!]

Yes, automation, AI, misconceptions thereabouts, IDSS, and the continuing importance of humans in the process, are all covered—quite often in contexts that matter in a social response-based, “Was*ISsy” sense. Although I found a few minor omissions or deficiencies in my read of it, this is about as good of a review article on the topic as reasonably can be done, given the limited space allotted.

Let the discussion begin! As a highly experienced, scientific severe-storms forecaster who was not involved (none currently in my unit were, to my knowledge), I’ll have a few tidbits to add too. There are so many places to go with this. I hope we do. Such discussion matters. I’ll post few early thoughts in multiple parts here.

Whither human understanding of the meteorological situation involved in a forecast? The article heavily emphasizes evolving educational requirements, and rightly so. It also alludes to the need for continuing meteorological and situational understanding on the part of forecasters (which begins, but does not end, in university education). The atmosphere offers continuing education, in its own language, throughout our careers as forecasters. It is professionally incumbent upon us to use observational data to read and understand the language of the atmosphere!

After all, regardless of how sophisticated the forecast models or post-processing are, we cannot optimally communicate a situation we do not deeply understand. Put another way, even the most eloquent and convincing communication of an unknowingly bad forecast, or incomplete understanding of the scenario, invites potentially tragic and economically costly consequences. Or as a colleague once put it, “You can spray-paint a turd gold, and it’s still a smelly turd.”

That means human diagnosis, by all available means (including hand analysis), still is important, and will remain so. Detailed, immersive situational diagnosis (subjective chart analyses, satellite imagery, radar imagery, soundings, radar-derived wind profiles, objective analyses, observational data of all available types) is absolutely foundational to every forecast scenario, but especially short-fused, multivariate, and high-impact ones, such as severe local storms and winter storms. Optimally thorough diagnosis is necessarily time-consuming and demanding of undistracted concentration in an operational setting. I’m glad the authors recognized and noted the importance of expert roles in forecast teams, and of fully attentive acuity. Institutionally, across public and private operations, we must reduce, not increase, distractions and disruptions to concentration on the forecast floor!

Given all the above, it is crucial for university and professional continuing education to maintain emphasis on meteorological situational understanding as the fundamental core of all forecasting and IDSS, and not lose sight of fundamentals in the zeal to embrace ever-more precise and prettier output. The safety of those who we serve with the forecasts is at stake!

More in forthcoming Part 2 and Part 3…

December 10th Tornado Devastation: Thoughts after Four Days

by Leave a Comment

Thoughts I’ve accumulated from last Friday night’s tornado disaster in the Mid-South, Tennessee Valley and Kentucky…

Forecasts were very good. Outlooks, watches and warnings ramped up into the event. Sure, one can look back and do “coulda shoulda” style Monday-morning quarterbacking about peripheral details or one level this way or that. Such second-guessing happens after most events, soonest from people who don’t have what it takes to “sit in the hot seat”, and instead make a social-media presence as nuisance trolls. Holistically, the Integrated Warning System did its job and did it well in every step until public safety (in a few locales). Mayfield, KY, had been in a tornado watch for over 6 hours before it was hit, and that whole track of tornado(es) was very well-warned by local forecast offices.

Were there glitches? Of course. The St. Louis-area office had to shelter temporarily when a tornadic mesocyclone nearly hit them — the same one that hit the Amazon warehouse in Pontoon Beach, IL, soon thereafter. The Paducah KY office lost power and the ability to send radar data for a few hours; their warning service was backed up quickly and adroitly by Springfield, MO (who has some experience with these things).

Footage from the most intense tornado damage areas presents awful scenes — reminiscent to me of historic events like Waco-53, Wichita Falls-79, Spencer SD-98, Tuscaloosa-11, Joplin-11, Moore, OK (thrice: 1999, 2003, 2013!), etc. The collapsed buildings and slabbed houses in particular will tell a terrible story. Despite good warnings, long-tracked tornado(es) from AR into MO/TN/KY were deadly in multiple places, not just Mayfield, and separately in that Amazon warehouse across the Mississippi River from St. Louis.

The morning after through today, it’s very important not go “instapundit” and rush to spouting exact figures yet for casualties, track length, or damage rating. Was the southern Kentucky event the longest-tracked ever? We don’t know yet; there’s some evidence both ways at this juncture. [Take some time to read the definitive, formally published science on the 1925 Tri-State tornado path.] How long-tracked & intense were all the tornadoes? We don’t know. All else is speculation. Until NWS damage surveys w/boots on ground are finished, we won’t know. The rest is variably educated guesswork. In such mass destruction, surveys and debris cleanup are still way incomplete. Casualty totals remain in flux for days, and some people perish later from injuries sustained (unfortunately, one was that beautiful Kentucky baby who initially survived in her car seat). Don’t treat any tornado info as final until survey results are released in days to come. Let’s back off, be patient & let the process fill in those facts with time.

Journalism on this event has run the gamut in quality from excellent to wretched, and that’s just in one newspaper (Washington Post). What else is new?

The worst of the journalism: take this steaming heap of dung, for example, sprinkling in some legit facts with factual errors, sloppiness and conjecture. Let’s look into this story a little closer. It was written by a political correspondent, not a scientist nor science reporter. So adjust your bias detectors & credibility expectations accordingly. More importantly…It also states, “The tornado that struck over the weekend that traveled from Kentucky into Arkansas…”. 1) Factually wrong. Backward. 2) Premature at story time on claiming that it was 1 (“the”) tornado. Surveys are not done yet. We don’t know. There goes the story’s credibility. If one can’t get basics like that right, what else do they screw up in any sort of stories? Don’t write “news” like this, kids.

Want an example of a well-done, well-researched, smartly organized, on-point story on the tornado event, authored by someone with solid credentials and impeccable subject-matter experience? Look no further than this excellent article by Bob Henson, which you really ought to read. Both this masterpiece and the junk story above appeared in the same newspaper.

The sad stories are worth reading too. Every tornado disaster has a human side, perhaps none more compelling in a tragic way than this story of the Kentucky baby who survived, bruised and conscious, only to perish later of a stroke probably induced by her injuries. I post that not to glorify tragedy, but to remind all that every single digit of every tornado death count involves the loss of someone and their future, someone who had friends and loved ones now grieving. In this case, the future was an entire lifespan. As a parent, how can this not hit hard? Sometimes we need these reminders, and we need to get uncomfortable. Detachment breeds complacency.

Let’s get brutally honest about something here. When a violent wedge tornado engulfs a densely populated area, devastation is assured. Even the very best warnings only minimize, not eliminate, casualties. Think Greensburg KS (4 May 2007), which had a similarly intense radar presentation as Mayfield, and excellent warning. We do what we can to minimize that from the meteorological end, of course. Many more vulnerabilities, after warnings are issued, still exist in the South, at night, with local preparedness, warning receipt, access to sturdy shelter, construction quality, etc. It’s complicated. As Bob Henson wrote in the above-linked good story: “…the timing of the tornadoes, coming in the dark of night, their exceptional intensity and the population density of the region hit were all key factors in the catastrophe — which advanced warning could not overcome.” That’s the bottom line.

Mayfield candle and Amazon facilities: It’s still to early to know a lot of specifics beyond the obvious fact that large-span factory/warehouse structures collapsed onto people and killed some. So let’s be patient on that and not start foaming at the mouth based on anecdotal interview stories. As a severe-storms forecaster concerned for decades with large-venue vulnerability & safety (including occupied factories), this bothers me…a lot. We must learn why so many perished there, not to assign blame but to make a safer tornado future. I’m very glad Tim Marshall is on-site in Mayfield, so we will assuredly get the best possible assessment there. From what is obvious, and in a general sense: if you want to look to improve on tornado casualties in situations like those, look to (lack of) local sheltering options, improved receipt of and response to both watches and warnings, and of course construction.

We can say already that this is a story of failure: structural failure. When roofs fall on people, crushing injuries happen. We know that caused many fatalities in single locations like these warehouses. This already presents us with two areas on which to focus, specifics to be determined:
a) Sheltering (such as sufficient capacity safe rooms and procedures to get workers into them fast and orderly) for existing large-span facilities that may never be retrofitted for decades and decades into the future, and
b) Engineering for new structures of that sort, along with safe rooms. New ASCE engineering standards are fixing to go into effect, which will be good, but we’ll still be dealing with non-compliant legacy construction hit by tornadoes for 200+ years.

That brings us to onsite safety in general. Here’s the encouraging news: We have a major positive example to use! In July 2004, a violent tornado demolished the Parsons factory near Roanoke, IL. Nobody died. Why? Facility management had a well-organized, practiced tornado-safety plan, sturdy shelters to put people in, and storm spotters to give them notice added to what they could get from official warnings. Here’s the scientific conference paper on that shining example of how to do it right.

We learned and advanced from Waco, Palm Sunday(s), April ’74, Wichita Falls, Spencer, Moore, Tuscaloosa, Joplin, etc. Now this. Over time we’ll learn from last Friday night’s horrors, so we can educate & try to keep it from happening again, and so their tragic loss isn’t in vain.

Communicating Forecasts for Dangerous Weather: Points and Counterpoints

by Leave a Comment

This entry is stimulated by Bryan Norcross’ guest essay in the Washington Post regarding the communication of uncertainty and determinism in the Florida forecasts of Hurricane Irma. Please read this if you haven’t already.

I respect Bryan’s perspective, whether or not I always agree, simply given his extensive experiences in trying to convey the hurricane-threat messages. While doing so somewhat bass-ackwardsly (1-in-1- vs. 1-in-5 or 1-in-3 “odds” as he states), he is arguing for probabilistic forecasting as a solution. Chuck Doswell would appreciate that. Here is a splendid and highly relevant essay by Chuck on weather-hazard decision-making in the face of uncertainty, and another on probabilities in forecasting. Those two pieces also should be “required reading” in this topic.

How those probabilities are translated or expressed don’t necessarily have to be as percentages, or even odds. They can be color fades, categorical labels translated from probabilities, and many other possibilities. It depends on the type of threat and the intended audience. Since intended audiences vary widely, even for the same hazard cause (in this case, hurricanes), and since this source offers multiple *types* of impact threats (wind, surge, heavy-rain floods, tornadoes), the possibilities are numerous. You really need to multiply the threats by the different types of audiences for the information to get the number of ways a forecast can (should?) be expressed. One size does not fit all!

A supply-side effect of taking such an exercise to its logical conclusion is of course a “herding cats” ordeal, where NHC becomes tasked with producing so many different kinds of graphics, texts, and forecast interpretations that the meteorology does suffer…and what good would that do anyone? Meteorological accuracy is at the heart of credibility; without it, all else is useless. Forecasters must not be forced to sacrifice that! Another risk is on the demand side: the “information overload” and head-in-sand or “Ostrich Effect” phenomena.

The bad news is that hurricanes, for their sheer complexity and layers of uncertainty and vast diversity of targeted “publics”, are about the toughest communications nut to crack, front to back. The good news is the same, because once we arrive at better solutions for communicating hurricane dangers, to the most people possible, in the most different effective ways possible, the less-complex weather-hazard producers (heat, for example, or even tornadoes), should be easier to solve from a communications perspective. That’s why I’m glad research efforts and discussion groups like WAS*IS exist, to help to hash these things out.

Still, as a former NHC forecaster and current severe-storms specialist at a different forecasting entity, who has observed closely these developments in the increasingly long time since I left NHC, I can attest that it the hurricane is a nasty hydra — just a big wet multi-hazard mess, tentacles swinging everywhere. Against it we have made amazing progress on the scientific side, but only fits and spurts in terms of COMMS.

Social media is another wild card Bryan didn’t even address, and brings with it a host of challenges in terms of getting the right forecast to the right audience, in the face of lots of rumors, fake forecasts, and non-credible sources.

Another aspect buried in Bryan’s essay, which I’m glad he addressed, is the “limits of science” issue. Excerpt:

    Just about every agency, company, outlet, TV station, website and app — including the National Weather Service, the Weather Channel, and most posts on social media — make explicit weather forecasts of a hurricane’s impact looking days in the future, well before that impact can possibly be known with specificity, based on the modern state of meteorological science.

    These misleading and confusing forecasts are produced by well-intentioned people and organizations because the formats of their text or graphics products demand it. There are seven-days worth of forecast boxes to fill in, so that’s what they do, even though everybody recognizes that the future weather when a hurricane is threatening is unknowable.

As a private-sector meteorologist colleague stated in an offline response: “Essentially we are being required to give a consistent, nearly perfect, 96-hour forecast for these situations because we want people safe (evacuate!) but we don’t want people to spend their scant savings unnecessarily. The problem is, that is beyond the state-of-the-art.”

In a nutshell, unreasonable demands for extended precision and accuracy often are made of forecasters, demands for information that lie beyond the state of the science. This hearkens back to something I have been preaching for years, with mixed effect:

We as meteorologists somehow must be willing and ready to state not only what we do know, but what we do not know.

The obstacle we face in doing so is the CYA cop-out mentality, “If we don’t do it, somebody else will”. Problem is, if the “somebody else” is offering “forecasts” that go beyond the reasonable state of the science, they are not credible! Yet that excuse has been used by managers, both government and corporate, to impose duties on forecasters that go beyond what the science justifies. One company produces temperature forecasts for specific days many weeks out — which is totally unjustifiable rubbish!

I am still not convinced, for a less-extreme example, that a day-7 or day-8 severe-storms forecast is consistently skillful or even useful the great majority of the time. Yet a group of forecasters is mandated to put one out every single night, needed or not, skillful or not, solely because the directives say so. Bryan rightly alludes to this dilemma. And on those (exceedingly rare!) occasions when the forecasters can alert to a hazard 9 or 10 days out? There’s no official means for it. Procedure and policy can handicap hazard communication, in this way, in the ways Bryan notes, and many other ways.

James Franklin, recently retired NHC Hurricane Specialist Unit chief, penned this response, also for Washington Post. The response struck me as overly defensive, even as he was making a valid point about the new “Key Messages” social-media product. Here is an example of that product:

I love this new offering, and wish Bryan had acknowledged it better. One valid question: is it enough? Another: what more can be done with limited time and staffing, on operational deadlines? The answers to those questions may compete and conflict! Bryan didn’t address that.

Still, while I understand and agree with much of what James wrote, I did not see Bryan’s article as an attack on NHC — least of all its forecasting — to the extent James did. Instead I saw it as a call for improved messaging of those forecasts, and not just by NHC! In that sense, Bryan’s piece serves a valuable purpose and is not “a solution in search of a problem”. “Key Messages” is a great step. But it’s not the end-all.

More can be done. More always can. But how, and by whom? Those are tricky questions without easy answers!

That improvement role need not reside solely or even largely with the forecasters themselves. The forecasters are educated and trained not as social scientists or graphics artists, but in this instance, as expert tropical meteorologists. They are the best in the world at what they do, bar none. I see the bulk of needed work — and this applies to other hazardous weather types too — as further along the chain of the Integrated Warning System. This means forecast translation and interpretation, both at the place of production and beyond, in media and emergency management, all the way to the responsibility the public audience has to understand what to do and have a plan in place.

Don’t expect the forecasters to do everybody else’s heavy lifting for them. Their job is to produce the most excellent predictions possible, and as I see it, they do.

Why can’t we as a weather science improve how we communicate hazards even more? Figure out additional ways those outstanding expert forecasts can be translated to even better utility once the data that drives the graphics have left forecasters’ desks. The same applies to severe-storms forecasts, as well as winter weather, heat, flooding, and all manner of weather threats. That is a positive goal and an outcome that I hope the meteorology and social-science communities can continue to work together to accomplish.