Great post, and I completely agree with what you're saying. All too often we treat forecasts as a specific number and celebrate when we beat them and flip out when we miss them. When in fact all of those experiences were within the range of possibilities. It leads to a lot of bad decisions. However, all too many business people have been trained for too many years to react to that specific number. Education is definitely required to help people understand statistics. I often use weather forecasts as a metaphor to help people understand. Of course, people always blame the weatherman for getting it wrong, but the reality is that weather forecasts can help you prepare for a range of possibilities that could happen. And that's really the point of a business forecast too.
Spot on Kevin. I'm going to creatively borrow (aka steal) that last line about the weather forecasts. It actually helps clarify the purpose of the forecast. "people always blame the weatherman for getting it wrong, but the reality is that weather forecasts can help you prepare for a range of possibilities that could happen. And that's really the point of a business forecast too."
The trap you're describing is subtle because it feels like diligence. The more precisely you model something, the more invested you become in that model being right, which means you start filtering out the signals that would tell you it's wrong. Precision becomes a way of managing anxiety, not a way of understanding reality.
In my work, I see this play out constantly: people come looking for specific answers because certainty feels safer than good judgment. But the answer they're looking for is usually the wrong question made very detailed. A rough framework with honest error margins beats a precise model built on false assumptions.
The same issue appears in economics, especially in how we measure inflation. While official estimates often include error ranges, there’s little effort to explore how different assumptions within those ranges would actually affect price adjustments.
That’s exactly what the NETs project aims to address, moving beyond reliance on a single figure and instead testing how varying inputs change the outcome. Nothing is taken for granted.
Great article as I anticipated when you first posted about it!
Great note on the margin of safety reframe at the end. Definitely a piece that is often skipped.
I'll add that that the sensitivity analysis can also reveal where the assumptions were lazy. If moving one input 5% blows up the outcome, that input very likely deserved a better conversation than it got. The model is not just showing the risk. It is also auditing how carefully each assumption was made.
Great post, and I completely agree with what you're saying. All too often we treat forecasts as a specific number and celebrate when we beat them and flip out when we miss them. When in fact all of those experiences were within the range of possibilities. It leads to a lot of bad decisions. However, all too many business people have been trained for too many years to react to that specific number. Education is definitely required to help people understand statistics. I often use weather forecasts as a metaphor to help people understand. Of course, people always blame the weatherman for getting it wrong, but the reality is that weather forecasts can help you prepare for a range of possibilities that could happen. And that's really the point of a business forecast too.
Spot on Kevin. I'm going to creatively borrow (aka steal) that last line about the weather forecasts. It actually helps clarify the purpose of the forecast. "people always blame the weatherman for getting it wrong, but the reality is that weather forecasts can help you prepare for a range of possibilities that could happen. And that's really the point of a business forecast too."
The trap you're describing is subtle because it feels like diligence. The more precisely you model something, the more invested you become in that model being right, which means you start filtering out the signals that would tell you it's wrong. Precision becomes a way of managing anxiety, not a way of understanding reality.
In my work, I see this play out constantly: people come looking for specific answers because certainty feels safer than good judgment. But the answer they're looking for is usually the wrong question made very detailed. A rough framework with honest error margins beats a precise model built on false assumptions.
Every time.
The same issue appears in economics, especially in how we measure inflation. While official estimates often include error ranges, there’s little effort to explore how different assumptions within those ranges would actually affect price adjustments.
That’s exactly what the NETs project aims to address, moving beyond reliance on a single figure and instead testing how varying inputs change the outcome. Nothing is taken for granted.
Great article as I anticipated when you first posted about it!
Great note on the margin of safety reframe at the end. Definitely a piece that is often skipped.
I'll add that that the sensitivity analysis can also reveal where the assumptions were lazy. If moving one input 5% blows up the outcome, that input very likely deserved a better conversation than it got. The model is not just showing the risk. It is also auditing how carefully each assumption was made.