To the extent that academic peer review is good for anything, it is optimized to catch honest mistakes. It is weaker against deliberate fraud and stubborn denial. Science has a presumption of fair play, a sense that the natural world isn’t a cheater. If you want to explain how a “psychic” operates, you’re better off asking a magician than a physicist.

Nearly two decades ago now, there was a dust-up when a couple French TV personalities got a clutch of physics and mathematics papers published, and even received PhD’s, and their “work” turned out to be nonsense. (The Wikipedia article on l’affaire Bogdanov is currently not terrible, and it contains more pointers to details than almost anyone could honestly desire.) The news stories about the incident really played up the “even the physicists can’t tell if the papers are nonsense or not” angle. That rather oversells the case, though. I read the Bogdanovs’ “Topological field theory of the initial singularity of spacetime” when I was a first-year grad student, and I could see through it. If you know what a Lagrangian is, and the fog doesn’t intimidate you, then you can tell something is wrong. If you don’t know what a Lagrangian is, you’re probably not reading theoretical physics papers yet.

So, what went wrong?

Nobody knows for sure, but we can hazard a plausible guess. Peer-reviewing is an obligation, and not a task that is often relished. It’s like jury duty: We understand the importance of what the system is trying to uphold, but on any given day, we’d rather be doing something else.

If you want to find the problems in a really bad paper, don’t ask an ordinary peer-reviewer. You’re better off with the scientists who congregate online to talk their subject because they have passion for it. And that’s what happened with the Bogdanov papers. The formal reviewers flubbed them, but sci.physics.research and the physics blogs delivered.

The old hands in the online forums will also be the ones most seasoned by claims of perpetual-motion machines and disproofs of relativity. They’ll have a more practiced critical eye and be less likely to get snowed by obfuscation. That long stretch of background exposition which sounds correct? It’s no guarantee that the new idea in the paper is — it could have been copied out of any standard text — and it can even be a warning sign. (We don’t need the 1001th introduction to why quantum gravity is a puzzle, or what the P versus NP problem is. Why do you, Mr. Author, think we do?)

The Bogdanovs had a pretty good success rate, getting six articles published. They also posted an un-refereed preprint that doesn’t appear to have been formally submitted anywhere (and whose mathematics was also wrong).

So, going by the numbers, physics — hardest of the hard sciences — must be six times as defective as postmodern philosophy, or Science, Technology and Society studies, or whatever it was that Alan Sokal demolished by getting his hoax paper into Social Text, right?

This is also a better success rate than Bohannon’s sting on open-access journals (157/304) or the “grievance studies” hoax (7/20, counting charitably). Of course, nobody who found the “grievance studies” hoax impressive thinks the Bogdanov affair exposes physics as rotten postmodern tripe!

(We might also note the irony that theme Sokal and Bricmont’s book keeps hitting is that the “fashionable nonsense” is gender-essentialist, tying femininity with menstruation for example. A good helping of queer and trans theory would harmonize perfectly with the stronger parts of their critique! Yet it is the people who appoint themselves their biggest fans who wish to erase queer and trans theory from the academy as just so much “grievance studies”. Likewise, I’d bet cash money that many Jordan Peterson fanboys regard the Sokal hoax as checkmate for postmodernism. But I’m having a hard time telling Lacan’s “This torus really exists and it is exactly the structure of the neurotic” apart from Peterson’s “I really believe that’s a representation of DNA.”)

I, too, once found Sokal’s prank basically the funniest thing ever. Taking on board what Sokal himself said shortly thereafter — “From the mere fact of publication of my parody I think that not much can be deduced” — I have to speculate, though, that the same point could have been made with more light and less heat even if his paper had been published with the journal’s contrivance, say as an April Fool’s item. The real point was not that the editors of one journal could be asleep at the wheel regarding one individual paper, but that many highly-regarded people had (apparently) said things about physics that were dubious at best, and without getting pushback from their academic communities. In retrospect, many years and two books from Sokal later, I cannot say that the prank aspect did a particularly good job of clarifying that. If anything, it seems to have convinced people of the merits of a glamorous but uninformative methodology.