Everyday we are bombarded by propositions. It takes discipline to critically examine them. For most of us, it is hard to steer the mind away from our gut reactions. If something feels right, but only vaguely, I try to ask
How would I know if this argument is wrong?
How would you?
You can compare them against known fallacies1, but that’s not always easy. You could appeal to a scientific analysis, but Science cannot arbitrate all propositions. Sometimes we have to make a pressing decision without knowing the facts. Sometimes we have to take a moral position2.
One thing Science does offer us is Method.
Bad Arguments can Overwhelm Us
Interrogating all the claims we hear is hard.
When confronted with ideas, Scientists often hedge their language. This can be frustrating to the uninitiated, especially when discussing policy or other applications of scientific research. This is cultural. One absorbs it while training in Science. You don’t want to inadvertently commit to a hypothesis before it’s been repeatedly tested.
When confronted with ideas, pundits tend make sweeping statements, pithy ones, easy for large audiences to digest. Without nuance, such statements made about Science are often wrong. Sometimes blatantly. When misinformation aligns with personal bias or emotion, it can go viral3.
Logically, sweeping statements should be the simplest to refute. You only need a single counterexample. This is how Nassim Taleb introduces the Black Swan. Negative evidence - ruling conjectures out - is crucial to the methodology of Science4.
But even sweeping claims take time to refute. Compared with ease of sharing bad ideas, an enormous amount of effort is involved in making counter arguments5. These discussions require nuance, which immediately hits the brick wall of cognitive bias.
For our own sake, is there a generic approach to quickly filter good arguments from bad?
Good Arguments Embrace Accountability
Karl Popper argues in The Open Society and its Enemies that the methodology of Science is distinct from other philosophical methodologies in that its arguments are rigid. Meaning is established by direct and repeated observation, not by reactive changes to language or discussion. Bad arguments are ended by falsification6.
The arguments I’m most concerned in today’s essay are those which similarly fail to be rigid, but can weasel their way forward in spite of the evidence. As Carl Sagan would argue7, hidden in these arguments are ideas that are not falsifiable. Rather, they are metrics or goalposts that can be rhetorically shifted beyond accountability8.
Good arguments should be falsifiable. Arguments that avoid accountability by construction should be a red flag.
Once such argument I’ve been thinking about a lot lately goes something like this:
“Nobody understands « insert subject here ».”
It’s probably one of the most accessible examples of an unaccountable argument. It also happens to one that we’ve also been discussion recently:
Who Understands Quantum Mechanics?
It’s a trope among physicists that “nobody understands quantum mechanics”. It’s often offered as a humorous introduction to the subject, to either students or a general audience.
Let’s explore a framework for judging whether and to what extent we collectively understand something.
The Johari Window
In a famous - or perhaps infamous - speech, then Secretary of Defense Donald Rumsfeld attempted to describe an application of the Johari window to the methodology of risk assessment.
In the traditional Johari window method, this was framed as a comparison of things about yourself known to you versus things about you known to others. In the context of risk assessment or skill acquisition, these quadrants take on slightly different meaning.
Let’s take Quantum Mechanics as an example. As you begin to learn, from your perspective, you are building up your second quadrant, the known knowns.
Aside from those, the unknown things come in many flavors.
There are the things that you know exist but don’t yet understand. You can glean these from the table of contents of your textbook.
There are things you don’t know exist and hence do not understand. The PCT theorem might be an example of this - it’s something not traditionally treated in an elementary course in Quantum Mechanics.
There are things that the community knows that we do not know yet. These might the results of specific experiments yet to be performed, or mathematical conjectures not yet proved.
Finally there are the things that nobody is even aware of, ideas that have not yet cross the community’s consciousness in anyway way.
Each of these items seem qualitatively different. How do they fit into the Johari window? 1 is clearly in the third quadrant, they are facts you know that you don’t know. 2, being made up of facts you aren’t aware of - but the broader community is - belongs in the first quadrant. Both 3 and 4 then belong to the category of unknown unknowns:
Your Johari Window
We can attempt to model the unknown unknowns by creating a similar matrix for the community as a whole:
The Community Johari Window
In that case unknown number 3 fits in the 3rd quadrant ( lower left ), and unknown number 4 belongs to the 4th quadrant ( lower right ).
What is tricky about this community construction is that the first quadrant is undefined9. The community as a whole cannot have knowns that are unknown by definition. This is fine, this is just an extrapolation of the Johari model beyond its intention. But it is a useful tool contrast with the personal Johari window.
Understanding certainly entails knowledge, so from the perspective of the community Johari window, the statement “Nobody understands Quantum Mechanics” would put Quantum Mechanics into which quadrant, exactly?
From my perspective, there are three possible answers.
If we take Feynman’s argument to mean that nature is just fundamentally stochastic, then we are solidly in the third quadrant - the known unknowns. We don’t know what path a quantum particle takes, it takes all of them10.
If we take a more normative view point, perhaps one like nobody understands Quantum Mechanics because nobody understands the measurement problem. Then we are squarely in the fourth quadrant - the unknown unknowns.
Finally, we could take a mystical viewpoint - that Quantum Mechanics is inherently mysterious and humans are just not capable of understanding it - then we’re in trouble. This implies that there is some truth that is just inaccessible to us, specifically. That entails that the truth exists, but we just can’t have it11. That’s the first quadrant - the unknown knowns. Individually, this is fine in principle, but collectively, this statement is undefined.
Now we can interrogate the claim.
When one asserts that nobody understands Quantum Mechanics, which of these three viewpoints do they intend?
Scientific Investigation and the Dunning-Kruger Effect
Empirical psychology suggests that knowledge is a prerequisite for the faithful assessment of said knowledge. On its face this is kind of obvious. To better gauge your own understanding of a topic, it helps to better learn that topic.
The Dunning-Kruger effect is an empirical “law” which asserts that individuals with minimal exposure to a topic - those who understand it the least - are the most overconfident about their understanding. It’s a fairly robust finding.

It’s worthwhile to apply this effect to the emergence of a new field of Science. When Quantum Mechanics first arrived, Physicists must have been fairly confident in their understanding of it - they certainly were confident enough to publish! Reading through Planck’s brazen use of mathematics to solve the ultraviolet divergence feels like a good example.
As time wore on - and more details came to light - those physicists became less certain. This uncertainty was clear from the philosophical consternation, like Schrödinger’s famous thought experiment.
Things eventually evolved, thanks in part to the steady hand of Niels Bohr
“Everything we call real is made of things that cannot be regarded as real. If quantum mechanics hasn't profoundly shocked you, you haven't understood it yet.”
Niels Bohr, (1958), The philosophical writings of Niels Bohr
These days we have considerably more knowledge about Quantum Mechanics than its founders ever did.
What’s amusing to me is how well the Dunning-Kruger effect comports with Popper’s approach to Scientific knowledge. Science advances via negativa, and as such, that which survives the test of time typically corresponds to what we have the most faith in. In other words, the more exposed a surviving Scientific result is to experiment, the more the would-be unknowns around it get peeled away.
The Dunning-Kruger Effect in Reverse?
Is the claim that Nobody Understands Quantum Mechanics even sensible?
The Dunning-Kruger effect refers to an overconfidence by the unskilled. Could it be that the unskilled are maximally underconfident of the skills of everyone else?
Let’s assume this contrapositive interpretation makes sense, and examine our three scenarios.
The Third Quadrant : The Feynman Viewpoint is semantics.
Feynman arguably understood Quantum Mechanics better than most. His interpretation that Quantum Mechanics involves a host of known unknowns is entirely consistent with expertise and the Scientific method. In some sense, it’s the minimalist version of the proposition that Nobody Understands Quantum Mechanics. Arguably it’s more of a humorous, semantic observation than a serious proposition.
The Fourth Quadrant : The Tautological Viewpoint is trivial.
Modern Quantum theorists might lament our poor understanding of the details of Quantum Mechanics. The nature of the wavefunction and its collapse. If we grant these experts their assertion, then nobody understands Quantum Mechanics is consistent with Popperian falsification. It is not the business of Science to understand, but rather to dig into the fog of unknown unknowns and retrieve something known from it. In short, Science is a process, and nature will always have unknown unknowns. This is a consistent viewpoint, but tautologically so. We don’t understand what we don’t yet understand.
The First Quadrant : The Mystical Viewpoint is an incorrect extrapolation.
Finally, as we have seen the mystical interpretation asserts the impossible12. In my experience, the ratio of serious physicists to armchair enthusiasts who hold this viewpoint is fairly low. Hence it seems likely to me that the mystics are highly unskilled and are therefore conflating their ignorance - their known unknowns - with the community’s ignorance - which for the skilled is of a different nature. This certainly comports - in spirit anyway - with the Dunning-Kruger effect.
So, when some asserts that nobody understands, we should ask them in response, in which sense do you mean?
Of Knowledge and Understanding
Learning can be hard, especially as an adult. Mastery is a struggle. It is overwhelming.
If you are learning things that are already understood, you often have access to a methodology. There are textbooks, videos and often, a tradition of teaching the subject. You can draw from all of these to quickly and linearly gain information.
But is knowledge of this information equivalent to understanding?
Stare at this question long enough, and it’s not hard to see where the Dunning-Kruger effect comes from. Without the crutch of a textbook, can you really say you know how ideas fit together. Can you predict, based one on chapter, the subject of the next? Mastery of a subject requires knowing only on the bounds of your knowledge but that of the community as well.
The fourth or lower right quadrant - that of the unknown unknowns - is almost by definition difficult to grapple with. To quote Rumsfeld
“There are things we know we know. We also know there are known unknowns; that is to say we know there are some things we do not know. But there are also unknown unknowns—the ones we don't know we don't know. And if one looks throughout the history of our country and other free countries, it is the latter category that tends to be the difficult ones.”
Sec. Donald Rumsfeld, DoD News Briefing, February 12, 2002
The methodology of Science requires a certain competency with ignorance. Expertise is developed - and perhaps defined - by the methodology used to interrogate the unknown in the absence of structure.
The instinct to invoke Scientific knowledge to resolve propositions of value can be morally fraught. It has also become commonplace. Philosopher Michael Sandel has written an excellent book on this matter. That said, we can certainly use the methodology of Science in discourse without misapplying its findings.
Pundits are not just limited to the talking heads on TV, or even “famous” influencers. A study published last year in PNAS suggests that “habitual users” play an outsized role in the spread of misinformation.
Philosophers will typically howl here that Popper’s work is incorrect. Amusingly, nobody actively practicing Science takes such wholesale disregard for Falsification seriously. As Feynman would say, “The philosophy of Science is as useful to Scientists as Ornithology is to birds.”
Here’s a sample of some recommended methodologies for refuting misinformation. It’s worth exploring them to see just how involved they can be.
While there is nuance to this method, Popper’s essential point was that the culture of Science causes our culture of individual liberty. Dispassionate analysis, empirical accountability, and pruning ideas that don’t work is how we maintain freedom from authoritarian regimes.
Sagan’s invisible dragon demonstrates how such alternative methodologies work in his book, A Demon Haunted World.
In The Open Society, Popper explicitly shows how Plato, Hegel and Marx are all practitioners of this kind of rhetorical methodology.
The pendants here might try to put something like Goldbach’s conjecture into this bucket. Things that we cannot disprove but also cannot prove. Thanks to Gödel’s incompleteness theorems we know that - given any specific set of rules - there can be true facts that we cannot prove from those rules. These could fit here, provided we modify the construction about to replace the word “known” with “true statement”. But that is against the spirit of what we mean by knowledge or understanding. So in our current framework they would first best in the third or fourth quadrant, depending upon our awareness of the existence of a particular proposition.
We spoke about this at length the other day while discussing Feynman’s Path Integral formulation of Quantum Mechanics:
Without, perhaps, divine revelation or something to that effect.
If instead you believe the mystics are just reframing the unknown unknown quadrant in the collective Johari window, then we are just going to bucket them with those who assert that nobody understands Quantum Mechanics yet, because we don’t understand the collapse of the wavefunction or some other specific feature of Quantum Mechanics.
This thought experiment is very interesting from a philosophical point of view. What the pandemic had taught us is to follow the science but to question the science every step of the way. By not doing so we are prone to take shortcuts around the scientific method into the realm of blind faith.
It looks like here too, the Substack App has failed to render some of the LaTeX diagrams. I’ll upload them as images soon. Meanwhile it should be clear using the web browser.