On the Structural Fragility of Science Communication

We explore the inverse relationship between the process of Science and its reporting.

Hey. It’s been a while. After a few bizarre pieces and publications near the end of year, I’ve been wrestling with what constitutes good Science writing. I was worried that the community was doing more harm than good. I spell out most of those concerns in this piece. It took me a while to pick apart what kind of writing I felt was causing harm and why.

The good news is that I’m more motivated than ever to talk about Physics, especially teaching it. So stay tuned for more!

Thank you so much for reading.

“People are always asking for the latest developments in the unification of this theory with that theory, and they don’t give us a chance to tell them anything about what we know pretty well. They always want to know things that we don’t know.”

R.P. Feynman, QED.

Escape from the Ivory Tower

I left professional physics in perhaps the most unprofessional manner possible. I left without telling anyone. One day I just deleted my email account, bought a plane ticket to the Pacific Northwest and walked out the door.

My colleagues and I were working on a paper that made no physical sense to me. None. We were studying the dynamics of general class of inflationary cosmology. More precisely, we were playing with models that could explain the inordinately small variability of CMB polarization at the largest scales in the sky¹. After a few ponderous discussions, we ended up with a catalogue for all possible scenarios for the dynamics that took place outside the observable horizon.

We were making a theoretical study of what could never be measured by experiment, in principle. Even if the implications were observable, the framing of the problem made me extremely uncomfortable².

I had written a four page simplification of our scenario, taking the perspective that what was outside the observable horizon wasn’t worth discussing. I raised the issue vociferously at our last meeting, but after a long pause the collective response felt something like “There, there young man. Thank you for your concern. We’re going to push ahead with this analysis anyway. It’s more interesting.”

Our Skype meeting ended at nearly 2:30am. I was in Asia. The rest of my collaborators were scattered between Europe and North America. I slammed my laptop closed and paced alone in the dark.

The meeting room was cold and empty, but the dim glow of the city lights reflected off the white office walls and linoleum floor. I took a long, hard stare out of the louvered windows at the other Academic buildings across the way.

I reopened my laptop and booked a flight.

Primordial Perturbations turned to Dust

String Cosmology was exciting precisely because it offered a window into otherwise inaccessible physics at extremely high energies. For example, if the aggregate polarization of photons in the cosmic microwave background came out a certain way, this would be evidence for “primordial tensor perturbations”, which were consistent with a wide range of models form String Theory.

For a hot minute, the BICEP Experiment reported the measurement of precisely those “B-polarization” tensor models.

That original announcement was a litmus test for me as a Scientist. My entire work in inflation up to that point - the bulk of my thesis research - had involved the study of small field models of inflation. Models that would be completely ruled out because they couldn’t generate those tensor modes.

As a new postdoc I had to smile at the news, and tell everyone in our group that I was wrong. I had to change my views “because that’s how Science works.” All that work, all those ideas, all that job hunting, branding and ego giving talks about those ideas. I had to throw it all away. I’m proud to say I was ready to do just that.

Amusingly, I didn’t have to. But the BICEP team did³.

Not long after the announcement of statistically significant, primordial B-mode polarization measurements, they combined their analysis with the Planck team and came up with nothing. The original observations were likely due to space dust. Noise, in other words.

That’s just how Science works.

Good Science Ages Well

Crazy, new scientific ideas are an important part of the Scientific method⁴. Science isn’t really in the business of labeling statements as true or false. Often that’s impossible to do in real time. Science about distilling what could be true from observation, and setting statistical limits to the applicability of those ideas.

Old Science - like Newton’s Laws of Motion or Maxwell’s Equations of Electrodynamics - have been around for hundreds of years. They have been subjected to an enormous number of tests, explicit and implicit. Refinements exist, but to within a well-defined range of error, these models of physics persist.

We still teach them. We still apply them. These are still good scientific ideas worth spreading.

By contrast, results from new Science - ideas or experimental results recently proposed or presented - are volatile. They should be viewed with skepticism until we have more checks and more data. Things might not pan out. They often don’t, as with the BICEP/Planck example above.

Shocks to our understanding - new ideas like those of Bohr, Einstein and Heisenberg, or even new results from Planck - can confuse us for a bit, but ultimately make our understanding of Physics stronger.

That’s just how Science works.

The Anti-fragility of Physical Science

This convex response of Science to volatile, new results is what Nassim Taleb has label anti-fragility. It is the opposite of fragility⁵.

It immediately follows, then, that Science Communication based on a “newsroom” model is vulnerable to that same volatility. It is fragile, because new Scientific results are less trustworthy than old ones. They frequently get overturned or revised. Just like the BICEP results.

That’s just how Science works, and it necessarily takes time. The problem is, that is literally the opposite of how a certain kind of Science Communication works.

Science News is structurally fragile; it is fragile by design. If we are looking to improve the public’s understanding of Science, using a newsroom model is manifestly fragilizing to the general public’s knowledge.

By its very nature, this kind of Science Communication is counterproductive

The “Wormhole Experiment”: a Case Study

In November of last year Quanta Magazine published a piece with a juicy lede: “Physicists Create a Holographic Wormhole Using a Quantum Computer.”

The main problem was that they didn’t.

With days, another paper appeared on the arXiv. Here, Norman Yao and friends zeroed in on the inappropriate use of the abelian subalgebra of the original SYK construction and demonstrating how their results on Google’s Sycamore Quantum Computer were likely indistinguishable from noise.⁶

A mistaken interpretation, in other words. Nothing nefarious.

These kinds of investigations move Science forward. The wormhole team weren’t wrong to publish the technical paper⁷. Creating a sparse version of the full SYK Hamiltonian to experiment on was a great idea. But should Quanta Magazine have given them such a glowing cover story? Did Lykken need to do a full-court press to popularize⁸ the result? Especially since they didn't release a preprint to the arXiv before submitting to Nature?

Quanta did publish at reasonable comment at the top of the story, and their follow up work has been commendable. But it is hard to imagine that many folks who saw the original headline also saw its refutation.

The next time something like this makes news, you can already see the “I thought they already did that?” or “I thought that paper was peer reviewed, how could it be wrong?” confusion in the general public.

Conflicting Incentives

There are plenty of reasons why a Scientist would want to see their work popularized. Public outreach is a significant component of a tenure application, and writing popular books is a common means to earn a bit of extra income. Also, virtually everyone enjoys a bit of notoriety.

Journalists and their Editors of course want eyeballs, and while many good reporters certainly aim to be correct⁹ they are also typically fully engaged in a fast-moving environment.

I am certain that all folks on the spectrum of Science Communication are passionate about fostering a well-informed public¹⁰. But I am concerned that anyone approaching it with a newsroom approach is doing more harm than good.

Trust in Science - which often collides with policy debate - is a contentious issue in today’s world. Members general public may not be hip to the nuances of the Scientific process, and reporting on results that will be quickly overturned seems to only drive the devise narrative that scientists “have no idea what they’re talking about.”

Fragilizing the Scientific understanding of the general public by emphasizing the new can only further erode that trust.

We simply must find a better way.

1. Just enough inflation: power spectrum modifications at large scales. Cicoli et al JCAP12(2014)030 ( https://iopscience.iop.org/article/10.1088/1475-7516/2014/12/030 )

2. To be clear, I have no problem working on mathematical problems in theoretical physics that have no bearing on actual experiment. It just happened that in this specific case we were looking link a certain kind of inflationary model with a specific, physical feature in the observational data. It was the framing in this context that felt disingenuous. Worse, I’m wasn’t sure that scanning through the options based on the standard, quasi-de Sitter approximation was appropriate, given that the low scalar perturbations were ultimately the result of a transition from the Bunch–Davies vacuum to the standard, Minkowski-like one.

3. For a complimentary perspective on this transition see the BICEP lead Brian Keating’s memoirs.

4. Assuming that they aren’t already ruled out by experiments and are precise enough to be tested or applied in some capacity.

5. The additive inverse if you like. However you want to define fragility mathematically, you can multiply it by minus one to get anti-fragility.

6. My personal concerned about the piece were two-fold. First, it was based on an SYK-model, which is manifestly an AdS_2/CFT_1 construction. While the ER=EPR conception of Susskind and Maldacena is theoretically compelling - particularly for N=4 SYM three spatial dimensions, AdS_2 has only one spatial dimension. As popularized in Edwin A. Abott’s Flatland, it’s hard to swallow the thought of a “hole” in one-dimensional space. That’s really just a breaking a line. It may be a topologists’ complaint, but it does make the headline feel like disingenuous hype.

Second, the approximation to the SYK model used by the experiment involved a restriction to an abelian subalgebra of the usual SYK algebra of operators. I didn’t immediately have enough sophistication to explain why that “smelled” off, but that is a considerable simplification of the original model.

It is perhaps worth nothing that the original authors have refuted the refutation, but not in a way that is convincing to this author.

7. Although of course not everyone shares this viewpoint.

8. See for instance, https://events.fnal.gov/arts-lecture-series/events/event/wormholes-in-the-laboratory-joe-lykken-fermilab-virtual/.

9. Their reputation depends on it!

10. Even if they’re approach or perspective on what constitutes good Science is flawed. See for instance: