Let the recriminations commence!

It's becoming apparent that the forecasts for a 'historic' snowstorm for New York city have proven to be incorrect.  At least one meteorologist has already apologized for not getting this correct.  So the recriminations will commence in the media, I'm sure.  Blogs will be written, Bill Belichick's nasty mischaracterization of weather forecasts will be resurrected, politicians will voice their displeasure, letters to the editor will be written, and so on.  Mea culpas and excuses will be offered.

All of this could have been avoided, of course, and I don't mean by the obvious decision to issue a different forecast.  Let me explain briefly from where forecasts come these days:  to an ever-increasing extent, forecasts are more or less coming from numerical weather prediction [NWP] computer-based models.  Human forecasters who have to make forecast decisions are presented with a number of different model forecasts, some produced by different computer models, and some produced by starting the same model with slightly different input starting conditions.  To some extent, the variability within the ensemble is a measure of the uncertainty of the forecast, but even the uncertainty is, well, uncertain!  This collection of different model forecasts is referred to as an 'ensemble' of forecasts, and at times, there can be considerable variation among the different forecast model results, as was the case with this event.  It was quite likely to snow heavily somewhere at some time - the problem was to pinpoint when and where.  A 'historic' snowfall in New York city would have devastating consequences! 

In reality, what is inevitably true is that all the model forecasts will be wrong, to a greater or lesser degree.  No human- or computer-based weather forecast has ever been perfectly correct, and none ever will be.  Uncertainty is inevitable, so the best any forecaster can do is to say "[some weather event, like a blizzard or a tornado] is likely to happen, and my confidence in that occurrence is [some way to express the uncertainty of that forecast, such as a probability]."  Numerous studies have shown that forecasters are actually pretty good at estimating their uncertainty; this is seen by their forecast reliability.  Reliability of a probabilistic forecast means that, given a particular forecast probability, as that forecast probability increases, the frequency of occurrence of the forecast event also increases.  Forecasts are perfectly reliable when the probability forecast is X percent and the occurrence frequency is also X percent.  In general, it's not a good idea to present forecast event probabilities as either zero or 100 percent, although it's possible in certain situations - the probability of having a blizzard in the ensuing 24 hours when it's presently 100 deg F in July is pretty close to zero.

If the forecast probability is neither zero nor 100 percent, then a particular forecast is neither wholly right or wholly wrong.  If the event occurs although the forecast probability was low (say, 1 percent), then this is the one case out of 100 you would expect to find.  If the event fails to occur despite a forecast probability of 99 percent, again, it's the expected one case out of 100.  In the case of the snowstorm in the northeast, the problem was to know just where the heavy snow would be.  Clearly, when the event occurs, you want the forecast probabilities to be high and when it fails to occur, you want the forecast probability to be low.  Of course, there at times when it doesn't turn out that way, as I've described.

Getting back to the 'dilemma' of choosing from an ensemble of forecasts, the most likely event is the average of all the ensemble members.  But even if that average actually turns out to be the best forecast, it will not be completely accurate.  And there are times when the forecast that would have been the best is one of the ensemble members - the problem is to know with confidence which would be the correct choice, and the challenge is that science simply can't predict that with accuracy.  And in some situations, none of the ensemble members is very close to the real evolution of the weather, unfortunately - i.e., none of the model solutions were accurate.  There is no scientific way to choose the "model of the day" and efforts to do so are simply a waste time!

Everyone wants the forecasts to be absolutely correct, all the time - but like the Rolling Stones say, "You can't always get what you want!"  We need to re-negotiate our contract with forecast users who expect us to provide a level of certainty we simply are [and always will be] incapable of providing.  Users deserve to know our uncertainty in the forecast products we send out.  Not doing so is scientifically dishonest - forecasters know their uncertainty, so not sharing it is to withhold important information!