Monday, April 18, 2011

Monday Morning Quarterbacks

How appropriate that I'm inspired to make this entry on my blog on a Monday morning. Some chasers have been commenting on the excellent performance of the Storm Prediction Center regarding the 3-day outbreak of severe weather starting on 14 April. Most of the posts are appropriately supportive of the excellent job done by my colleagues there over the past several days. But there always seem to be some trolls who want to play Monday Morning Quarterback (MMQ). This is hardly the first time I've seen such comments -- today's were only the latest in a long line of such over the years. Many people, including some storm chasers, seem to delight in finding fault with the SPC. Thus, I want to take some time to make a few things clear:

1. SPC forecasts are not only concerned with events of interest to tornado chasers. They include all forms of NWS-recognized "severe thunderstorm events": not just tornadoes, but also convective wind gusts and large hail. The SPC forecasts are not tailored specifically to the needs and interests of tornado chasers. If you're going to criticize the SPC forecasts, please keep this in mind. If you think you make a better chase forecaster than the SPC forecasters, then by all means -- define your forecasting criteria, post your daily chase forecasts in advance for a whole year, make the whole data set public, and show us the verification results for your chase forecasting. Note that successful chases can be associated with what amount to busted forecasts, so your tornado videos are not evidence of your forecasting capabilities -- only of your chasing capabilities!

2. The task of putting out severe convective weather forecasts for the entire USA, 24/7, is not a simple job. It's quite challenging, and I challenge anyone to produce the suite of products the SPC produces during the day shift every day -- products issued several times per day and others as needed -- for a whole year. Lest anyone think that the advent of numerical models and other objective forecasting aids has made this task easier -- think again! All the SPC forecasters put their names on all the forecasts as a matter of public record. After you've published your forecasts, then do an objective, statistical verification of them along the lines of the SPC verification programs, and compare your results to those of the SPC for that year. If you can provide such evidence that yours are even close to their results, then I might give you some credibility. If you can't exceed their results, then my recommendation is to shut the hell up!

3. The SPC suite of products is, like all other forms of weather forecasting, not perfect. It's likely impossible to put out an indisputably perfect forecast, so that's not even a worthy topic of discussion. The Facebook troll I encountered today made the statement that SPC forecasts are "wrong as often as they are right" -- thereby establishing beyond doubt that he doesn't have a clue about the subject. He was a legend only in his own mind, incapable of backing up his claims, naturally.

I'm far from blind to the shortcomings of the NWS, as anyone who has read even a fraction of my Web content can determine for themselves. But for a few years, I sat in the hot seat of the forecasters at the SPC (back when it was known as the SELS Unit of the National Severe Storms Forecast Center in Kansas City, MO) and, with all my education, I just managed to not embarrass myself as a severe weather forecaster. I wish I could convey the feeling of ignorance that swept over me as addressed a blank map of the US, onto which I had to produce my first ever convective outlook. Anyone who hasn't done this simply has no clue what it's like, or how challenging it can be. Even on those days when the SPC forecasts make significant errors, these forecasters are not just lollygagging around, spouting objective guidance-driven platitudes. They're doing a difficult job well every damned day! They're the best severe convective weather forecasters on the planet, so far as I'm concerned, until someone can step up and show me solid evidence that they're better than the SPC!

4. The fact that I respect the job being done by the SPC doesn't mean that I think they're perfect and metaphorically walk on water. They themselves would readily admit to shortcomings, I'm sure. When they make forecasting decisions, some of them turn out to be mistakes. The state of the science is far from complete and if you want to accomplish something useful to push forward the state of the science, I'm sure the SPC forecasters would be among the first to applaud your contributions. By all means, share your insights via conference presentations and publications in the formal meteorological literature. But if all you can contribute is after-the-fact MMQ'ing, then I suggest you shut the hell up!

A few folks with the capability to meet or exceed SPC forecasting standards might exist, but they're not doing the job right now that the SPC does. Until someone has done that job, they still have something to prove. If you can't produce the evidence, your claims to such capability remain undemonstrated and you have no qualifications to play MMQ!


===== Roger ===== said...

Wow. Outstanding! There's not much I can add to that except the simple word "thanks". You've hit the problem of arrogant second-guessers squarely and surely with a forceful hammer on all four counts! Next time some self-proclaimed expert crawls out of the walls and touts his superiority to those who have years and years of specialized training, experience and expertise in severe storms forecasting, I'll just direct them to this piece.

I'll be waiting to see if any of the blathering MMQs take you up on your forecasting an verification challenge. I'm glad food, drink or money isn't on the line; I surely would perish or go bankrupt while waiting.

Eric Weaver said...

In any field of human endeavor, there are those brave folks who do their best and take a crack at producing something of value.

Then there are always those who say they could have done it better, but who actually did nothing at all.

There is but one thing to say to the first bunch, even if their best efforts fall short: "Thank you for your hard work."

Similarly, there is but one thing to say to the second bunch: "Piss off."

The more we realize that whatever the second bunch says is of no real consequence, the better off we are.