Beware the Narrative Fallacy
Backpropagation for N = 1 is both natural and a fool's errand. Look forward instead.
How’s the Weather?
There’s a stark asymmetry to the availability of weather data. Getting a prediction for tomorrow is easy. Finding the prediction from yesterday? Much less so.
This asymmetry has bothered me since childhood. While delivering newspapers by bicycle, my gaze fell on the headlines or the weather predictions, depending on which side of the papers landed upwards in my handlebar basket.
I didn’t grow up in a Scientific household - if anything I’m from a family of journalists. Yet, I was always curious about the weather prediction’s track record1. Scientific inquiry, I was told, came naturally to me.
And so that was the story I told myself.
Of course, there are reasons for the asymmetry in weather data. First, there’s no practical utility in last week’s weather predictions. Second, weather is notoriously difficult to predict, so why advertise your shortcomings? Finally, despite this, we trust that those doing the predictions to get it right most of the time.
The validation of weather predictions falls outside the daily scope of a newspaper. There’s nothing wrong with that. We do not have time to apply the methodology of Science to every aspect of our lives. As long as a hurricane isn’t headed our way, we’ll probably be fine.
Narrative and the Illusion of Control
Every so often I discuss weather forecasting with other armchair meteorologists. There is one thing I hear a lot:
“Did you know that a 70% chance of rain doesn’t mean a 70% chance of rain?”
In this case I’ll just pass a confused stare.
“It means that 70% of the land in the area will experience some rain.” They reply.
I’m always amazed how this subtle shift in framing amazes others. Perhaps because it offers some insight into the forecaster’s methodology. But the practical difference between the two perspectives is irrelevant. It’s just a story2.
The human mind craves narrative so strongly it will impose it from behind3. The problem with that strategy is painfully clear from the front. A false or coincidental narrative damages our ability to predict4. While easier to understand, narratives negatively impact our practical understanding.
Often we worry about the unanticipated, improbable events. Especially those that can be damaging. But today in particular I’m concerned something more basic: an unfavorable outcome which had a reasonable chance of happening.
We struggle with the interpretation of randomness. Our internal responses to chance are limited. We understand the indifferent (50/50) and the nearly certain (99/1). But even then, context shapes human reaction to uncertainty5. Emotional investment in the outcome will further distort our perspective: the result of a 50/50 event can feel as thought it were 99/16. We may attempt to explain away the outcome of a coin flip as destiny. The narrative of inevitability feels better, especially if we can tie it to a hero or a villain. It absolves us - collectively or individually - of responsibility.
Put differently, prediction naively suggests understanding. Narrative steamrolls our empirical understanding to give the illusion of control7. We feel better about it. Chance is both ubiquitous and disconcerting.
But Science requires comfort with ignorance.
“It is our responsibility as scientists, knowing the great progress and great value of a satisfactory philosophy of ignorance, the great progress that is the fruit of freedom of thought, to proclaim the value of this freedom, to teach how doubt is not to be feared but to be welcomed and discussed, and to demand this freedom as our duty to all coming generations.”
Richard Feynman, The Value of Science
Damn The Weather!
Agency is a better framework than destiny or control. Strategy based on probability is susceptible to the narrative fallacy. A better approach is a strategy based on outcomes. Being able to react to changes on the ground rapidly is helpful. A strategy that is resilient against the unexpected can give you the room to maneuver. Better still is a strategy that benefits from unexpected shocks, an anti-fragile approach.
For many, today represents a transition from anxiety of what could be to the anxiety of what is. Please don’t attempt backfit an explanation to what happened. Don’t listen to those who do. They’re probably wrong and such sentiment is also unhelpful.
Calm your anxiety with action. Complaining and worrying won’t accomplish much. Ask yourself, how can we leverage this outcome for good?
Does it seem that impossible to you? If so, please remember that Science is all about aggressively trying to prove yourself wrong.
I grew up on Windward Oahu, where the weather is fairly stable, so this observation is even more amusing to me.
The careful reader might complain: “You can imagine a situation where within a given geographic area, some parts of the land have a higher propensity for rain, meaning that 70% will be unevenly distributed - near the mountains, say, or in a particular gulch.” But you see, you’ve just put a narrative down.
As Crichton pointed out, journalists - or at least opinion editors - are notorious for this. By the way, the linked quote comes from a speech he gave back in 2002.
This is a major theme in Taleb’s The Black Swan and can be studied from within Daniel Kahneman’s study of decision making. Any mention of Kahneman’s book should probably come with some recent context around the replication of psychological experiments, even the most empirically driven ones.
Almost every place I’ve ever lived has had a similar, “uniquely” local anecdote: “Don’t like the weather here? Just wait five minutes.”
Narrative in particular can distort our expectations of how representative a given occurrence is. These expectations are directly tied to our confidence.
Even if “control” in that context implies a total lack of agency. This is one opportunity for complaining and blaming emerge.