tag:blogger.com,1999:blog-2759913671101666257.post7566770556809328592..comments2023-05-24T06:02:06.480-05:00Comments on Chuck's Chatter: Let the recriminations commence!Chuck Doswellhttp://www.blogger.com/profile/03099345055614900157noreply@blogger.comBlogger13125tag:blogger.com,1999:blog-2759913671101666257.post-38370050330397600012015-01-30T11:56:51.845-06:002015-01-30T11:56:51.845-06:00Jason Goehring,
If I'm misunderstanding your ...Jason Goehring,<br /><br />If I'm misunderstanding your comments, it's because their verbal expression isn't conveying your intended meaning clearly. To this point, you've not managed to advance a coherent, logical argument that justifies your stubborn insistence that providing uncertainty information in weather forecasts is not the right thing to do. Rather, you simply keep repeating your flawed understanding of probabilistic forecasting<br /><br />Therefore, I conclude that this exchange is of no further value here, as we're simply talking past each other.Chuck Doswellhttps://www.blogger.com/profile/03099345055614900157noreply@blogger.comtag:blogger.com,1999:blog-2759913671101666257.post-72259802654483538242015-01-29T22:23:42.688-06:002015-01-29T22:23:42.688-06:00I think you are misunderstanding my comments. I&#...I think you are misunderstanding my comments. I'm not saying all probability forecasts decrease credibility. Putting up a 10% CYA probability often, when the forecaster is pretty much sure nothing will happen, reduces credibility. By saying 10% chance of rain for the next 3 days, a lot of the public just remembers "rain"; then it doesn't rain (what the forecaster really meant). So when there is a 80% chance of rain, the public says "they said it was going to rain those 3 days and it didn't" so they dismiss that chances. The biggest problem is in the public's understanding of probabilistic forecasts. Maybe instead of 10% chance of rain, 90% chance of sunny skies.<br />I'm also not saying that we need to treat the routine days with less than our best effort.<br /><br />The only other thing I'm going to add is this; some other industries that issue forecasts don't add probabilities. (Example) a few weeks ago I was going to sell some of my stock of soybeans, but then I read and heard 2 different market advisors (forecasters) say that they think the price will go up. That was their opinion, I knew they were not certain and there was risk. So I waited. Now I still have my soybeans, but its worth $3K less than it was 3 weeks ago. These market forecasts are just like weather forecasts, yet like you said weather forecasters are the butt of jokes and the market advisors are not humiliated when wrong. I know next time they will be right and I will make an extra $6K.Jason Goehringnoreply@blogger.comtag:blogger.com,1999:blog-2759913671101666257.post-22609962071255745882015-01-29T14:32:34.719-06:002015-01-29T14:32:34.719-06:00Jason Goehring,
You seem to be saying all probabi...Jason Goehring,<br /><br />You seem to be saying all probability forecasts are a CYA exercise. That's so wildly incorrect, I hardly know where to begin to explain. Just how do probability forecasts reduce credibility? <br /><br />Weather forecasters are the butt of many jokes precisely because the public realizes we can be wrong and are so more often than we like. It seems to me that an honest admission that we can't be exactly right all the time would increase our credibility, not decrease it!<br /><br />I don't want forecasts that make decisions for our users, who need to make those decisions for themselves using information in addition to the weather forecast that we know nothing about. All we do by pretending we can make perfect deterministic, binary (yes/no) forecasts is set ourselves up for humiliation.<br /><br />And the idea of treating routine forecasts with less care than big events is muddled thinking. We need to do our best job forecasting every day, not just days with potentially big events. Your comment suggests that our forecasts are only important on days with disaster looming, but that's just not true. Chuck Doswellhttps://www.blogger.com/profile/03099345055614900157noreply@blogger.comtag:blogger.com,1999:blog-2759913671101666257.post-28667677239292497532015-01-29T09:48:13.351-06:002015-01-29T09:48:13.351-06:00Chuck,
I agree that a lot of people want the forec...Chuck,<br />I agree that a lot of people want the forecast to make the decision for them. I also am not a fan of the words "likely, probably...". What bothers me is when a 10% or even 20% chance is thrown into the forecast as a "cover my butt" qualifier. It reduces the credibility of the forecast for real threats.<br />In my forecasting class in college, we squared the error, so I figured out it was better to be a little off all the time than far off some of the time. I had the best score, but looking back now, my forecasts were all junk. Didn't provide useful information.<br />I think forecasts have become more of that "little off all the time" mentality.<br />I think we need to convey the importance of the major events better and not stress as much about the little ones.Jason Goehringnoreply@blogger.comtag:blogger.com,1999:blog-2759913671101666257.post-41758556462525836142015-01-28T12:36:55.978-06:002015-01-28T12:36:55.978-06:00John,
What I have seen regarding the ensembles i...John, <br /><br />What I have seen regarding the ensembles is this: the dispersion in the single-model ensembles is not sufficient to support the idea that the spread is directly interpretable as uncertainty. Multi-model ensembles are better at getting the necessary disperson, but even they fail sometimes.<br /><br />No doubt the ensembles remain limited in their capability to describe the uncertainties, but they are clearly an improvement over relying on a single run of a single, deterministic model. Yes, the uncertainty is uncertain, especially at longer ranges. Thus, your confidence should be correspondingly lowered at those ranges. Chuck Doswellhttps://www.blogger.com/profile/03099345055614900157noreply@blogger.comtag:blogger.com,1999:blog-2759913671101666257.post-87694607736421169912015-01-28T12:30:52.823-06:002015-01-28T12:30:52.823-06:00Jason Goehring,
I agree that we have failed to ex...Jason Goehring,<br /><br />I agree that we have failed to explain the meaning of probabilistic forecasts in terms the non-meteorologist can understand. Unfortunately, your perspective concerning probabilities is flawed. Many studies have shown that using words such as "likely" have different meanings to different people. Probability as we are using it is a measure of our confidence in the forecast - I'm sure you realize that we cannot make forecasts with absolute 100 percent confidence very often. Do you want to know quantitatively how much confidence we have in the forecast, or do you want to guess that confidence level on your own? In order to use forecast probability as part of your decision-making as a farmer, you have to account for many factors in a decision that we know nothing about. Do you really want us to make your decision for you by pretending to be able to do something that you and I both know we can't deliver?Chuck Doswellhttps://www.blogger.com/profile/03099345055614900157noreply@blogger.comtag:blogger.com,1999:blog-2759913671101666257.post-90922596639377805782015-01-28T12:24:35.041-06:002015-01-28T12:24:35.041-06:00Jim Duke,
What you seem to be saying is that it&#...Jim Duke,<br /><br />What you seem to be saying is that it's better to err on the side of caution when it comes to devastating events. Whereas that may be true for decision-makers using a weather forecast as input, I would like the forecasts to not be biased one way or the other. Any bias undermines the credibility of the product.Chuck Doswellhttps://www.blogger.com/profile/03099345055614900157noreply@blogger.comtag:blogger.com,1999:blog-2759913671101666257.post-29889152802868199262015-01-28T10:51:17.131-06:002015-01-28T10:51:17.131-06:00Chuck,
Do you know of any research that has shown...Chuck,<br /><br />Do you know of any research that has shown that a forecast with little spread among the ensemble members is likely to be more accurate than a forecast with great spread? I know it seems to be a logical conclusion, but only if the ensemble members span the space effectively. That is sometimes not the case - especially as the forecast hour extends beyond 24-36 hours (which is the period that decision-makers want to start hearing about upcoming threats with confidence). There are times that we simply do <b>not</b> know the uncertainty inherent to our forecasts.John Harthttps://www.blogger.com/profile/04010160784572007095noreply@blogger.comtag:blogger.com,1999:blog-2759913671101666257.post-34393385064328470362015-01-28T10:16:17.675-06:002015-01-28T10:16:17.675-06:00I'm a meteorologist and a farmer. By adding a...I'm a meteorologist and a farmer. By adding a "percent chance of uncertainty" to the forecasts, they loose their usefulness. <br />For example, if somebody asks "is 3 days from now good for a picnic?" If I say "there is a 40% chance of rain, partly cloudy to overcast, and 5 to 20 mph wind"- does that help that person decide to plan a picnic? If I say "it will be a mostly sunny, light wind and will not rain until 9 pm"- then that helps that person decide.<br />Yes, I have a greater chance of being wrong, but that person has useful information. They understand that weather forecasts are uncertain.<br />If the forecast is 20% chance of rain. To a meteorologist, that means most likely it will not rain but there is uncertainty. To an average person, it means its going to rain.<br />Forecasts need to balance that uncertainty percent with usefulness. In my opinion, public forecasts are becoming more percentage and less useful.<br />Either way, we are going to be wrong with our forecasts in the minds of the public, mostly because they only remember parts not the whole thing. Just how much do you want your audience know what to be prepared for.Jason Goehringnoreply@blogger.comtag:blogger.com,1999:blog-2759913671101666257.post-57161484448203717612015-01-28T06:43:57.233-06:002015-01-28T06:43:57.233-06:00The key is "what we can provide". Witho...The key is "what we can provide". Without capability to fulfill them, the user's needs and wants each are irrelevant. Similarly to something I stated in an e-mail to which Chuck was party: <br /><br />We as an enterprise need to grow a pair and tell the "public" and everybody else who demands impossibly deterministic forecasts:<br /><br />"No. You're not getting this from us because it's not possible. Anybody who tries to sell you a forecast like this is trying to rip you off. The science isn't there yet to give you this rigid, specific answer. Nobody is going to be that good in your lifetime or mine. Here's the best we actually can do...(forecast + expression of uncertainty)"<br /><br />Alas, very few manager/bureaucrats (in government, and private-sector managers too) have the stones to be that honest with the constituency. We all pay for that in lost credibility and undeserved bad reputation.===== Roger Edwards =====http://www.stormeyes.org/tornadonoreply@blogger.comtag:blogger.com,1999:blog-2759913671101666257.post-60771511068503206702015-01-27T15:26:25.700-06:002015-01-27T15:26:25.700-06:00The NYCity snowstorm, like many other weather occu...The NYCity snowstorm, like many other weather occurrences, has a dimension beyond occur/not occur - impact. Some weather phenomena (tornadoes, violent winds, ice storms, etc.) threaten a level of human and economic disruption that transcends simple occur / non occur. Hence, the forecaster in the present instance can find shelter in the notion that the threshold for tipping into severe warning mode with dramatic and perilous calls to action is weighted lower because of the potential for disaster.<br />Jim Dukenoreply@blogger.comtag:blogger.com,1999:blog-2759913671101666257.post-29529785970570843822015-01-27T15:08:48.545-06:002015-01-27T15:08:48.545-06:00Rob,
Yes, I have thoughts about that, but not spe...Rob,<br /><br />Yes, I have thoughts about that, but not specific ideas on how to communicate uncertaity. I'm a meteorologist, not a psychologist or communications expert. Ideas are cheap. What we need is a process involving social scientists of various sorts to help us find out the most effective ways to communicate uncertainty. If we change the format/content of our forecast products, we should conduct a massive campaign to make users aware of the changes BEFORE we implement them and help them make the transition. This must include a way to get across the very idea that our forecasts are inherently uncertain, since we historically have kowtowed to what users want, not what they need and what we can provide.Chuck Doswellhttps://www.blogger.com/profile/03099345055614900157noreply@blogger.comtag:blogger.com,1999:blog-2759913671101666257.post-71198241703557718452015-01-27T14:05:58.275-06:002015-01-27T14:05:58.275-06:00Any thoughts on "how" to tell the users ...Any thoughts on "how" to tell the users that uncertainty? Snow ranges were used but those didn't work out well.Rob Dalehttps://www.blogger.com/profile/00132449143586969553noreply@blogger.com