Thoughts on decision-making in the face of uncertainty

Many people struggle with the notion that weather forecasts are uncertain.  They have to make binary decisions in the course of their lives:  go on that picnic, bring that umbrella, pour that concrete - or not do those things.  Weather plays a role in many such decisions, and people seem to know that forecasts are not perfect and never have been, but they persist in being upset when they make a decision based on weather that ultimately doesn't pan out, at least as they understand the forecast.

Perhaps I'm oversimplifying this, but it seems to me that the real challenge with decision-making in the face of uncertainty is the absence of accurate uncertainty estimates.  If weather forecasts are always wrong, you could always do the exact opposite of what the forecasts say and have it work very effectively for you - a permanently, completely wrong forecasting system would be just as valuable at a permanently, completely right forecasting system!  A forecast need not be perfectly accurate to be of value to users!

Of course, no weather forecasting system is perfect and there never will be such a perfect system.  If you know the uncertainties in the forecast, there are techniques by which you can manage your decision-making so as to optimize your results.  That optimization incorporates knowledge of both the losses experienced associated with not taking some action and having that weather event actually occur, and the cost of taking that action.  This is called the cost-loss problem.  If the cost of taking some action to prevent losses from some weather event exceeds the losses if the event occurs, it makes no sense to ever take such an action.  Different circumstances demand different decisions.  Optimizing the results of your decision-making requires you to have knowledge of your costs and losses, in addition to an accurate estimate of the uncertainties.

Sadly, it's well known that people often have difficult with knowing the true risks associated with hazards.  For example, although tornadoes are very scary to many people, the reality is that the probability of being killed in a tornado is pretty low.  There are much greater risks associated with, say, food poisoning in fast-food restaurants, or driving motor vehicles.

Most people struggle with understanding the probabilities related to weather uncertainties but they have a reasonable idea of some uncertainties.  For example, uncertainties tied to their jobs are usually more or less familiar to the workers.  If your work involves manufacturing something, you usually know about the likelihood of producing a defective product.  Similarly, uncertainties related to your home are often reasonably well-understood.  You generally know something about the chances that your water heater will fail.  When the uncertainties pertain to the weather, most people generally have no idea what those probabilities might mean and how to use them to make choices.  It's not that they need to know abstract probability theory to begin to grasp what weather probabilities (the language of uncertainty in weather science).  Most forecasters never did very well in probability and statistics!!  But people can use a concept effectively even when they don't actually follow the abstract mathematics.  Card counters in blackjack are making effective use of their knowledge of uncertainties - so well that casinos don't allow card counters to play!

Part of our problem is that traditionally, weather forecasts have not been expressed in probabilistic terms.  The use of probability of precipitation (PoP) was introduced in the mid-1960s but there never was any sort of public information campaign to help forecast users understand them - an awful oversight!  Curiously, even many forecasters don't know the proper definition of PoPs, although with experience and some feedback, they can become very adept at estimating their uncertainties in terms of PoP.

The end result of having little or no understanding of weather forecast uncertainty - and all forecasts are uncertain to a greater or lesser extent - is that forecast users will develop all sorts of heuristic methods for making choices.  Many of these are likely to be rather less than optimal use of the information the users have.  And apart from PoP and some severe weather forecasts, uncertainties are not mentioned in forecast texts and broadcasts.  Since that information is known, at least in the minds of the forecasters, this amounts to withholding needed knowledge from the users!  If we don't include the uncertainty information in some form, users must guess about that uncertainty, and their guesses often are wildly incorrect, such as thinking that forecasts are "wrong" the majority of the time.

Users can handle decision-making in the face uncertainty only when they know the uncertainties reasonably well from being familiar with them.  Unfamiliar uncertainties (as are those in weather forecasts) are inevitably mysterious and are a source of anxiety in decision-making as well as the source for cynicism about the forecasts.  People demand, unreasonably, that forecasts express the weather in binary terms - this event either will or won't happen - even though they must already know forecasters can't do that very well all the time.  What they evidently want is for weather forecasts to make their complex decisions (including much information that forecasters can't know anything about) for them.  Forecasters simply can't and shouldn't do this. We need to help our users understand more about our uncertainties or this situation will never improve, and users will continue to get less value from weather forecasts than what forecasters are capable of providing.