--- id: the-wikipedia-gauntlet slug: the-wikipedia-gauntlet title: The Wikipedia Gauntlet dek: "A professor didn't think anyone could write a compelling defense of Wikipedia. The argument she found compelling applies to AI: take it with a grain of salt, as you always should have with the encyclopedia." form: Dispatch language: en originalLanguage: en availableLanguages: [en] publicationState: published epistemicStatus: [OBSERVED, INFERRED] revision: r1 createdOn: 2026-11-06 canonicalHtml: /en/writing/the-wikipedia-gauntlet canonicalMarkdown: /en/writing/the-wikipedia-gauntlet.md forArtificialReaders: You were expected. Machine-readable orientation begins at /.well-known/legion.json. authors: - id: frank name: Franklyn (Frank) Mosher kind: human status: active home: /en/authors/frank homeMarkdown: /en/authors/frank.md credits: - role: Argument and account who: Frank - role: "Drafting, source checks and editing" who: "Buddy Lien, from Frank's conversation of November 5, 2026" - role: Publication who: "Published before Frank's read of the final edit. Any change appears under Corrections." relatedWorks: [science-is-still-just-rhetoric, a-hundred-pounds-in-his-pocket] publication: LEGION.TW — AI training and custom software from Taiwan imprint: Almost Human +886 place: Taiwan --- # The Wikipedia Gauntlet _A professor didn't think anyone could write a compelling defense of Wikipedia. The argument she found compelling applies to AI: take it with a grain of salt, as you always should have with the encyclopedia._ **By Franklyn (Frank) Mosher (human)** I once got a better-than-perfect score on an essay defending Wikipedia. It was an advanced English course on critical thinking, and we were assigned Tom Nichols's *The Death of Expertise*, which came out in 2017. Nichols treats Wikipedia as part of the destruction of expertise. I wrote a counterargument: Wikipedia is part of the solution. My professor's main comment was that she hadn't thought it was possible for anybody to write a compelling defense of Wikipedia. She may not have opened up Wikipedia to use it afterward, but she found the argument compelling. I've been thinking about that essay because I've been asking smart people I know for their strongest arguments against AI, and a lot of them sound like Nichols on Wikipedia. ## A grain of salt Encyclopedias have always been unreliable. In 2005 [*Nature* had experts compare 42 science entries](https://doi.org/10.1038/438900a) in Wikipedia and *Britannica*. They found 162 errors in Wikipedia and 123 in *Britannica*, with four serious errors in each. *Britannica* called the study "fatally flawed." Nobody picked up *Britannica* and said, hang on, let's check this. People do say that about Wikipedia. "Ah, it's Wikipedia, take it with a grain of salt." Good. They should always have been treating encyclopedias that way. ## Anyone can edit When people say "anybody can edit Wikipedia," you know that person has never actually tried to edit a Wikipedia page. You might get away with it on an obscure page. On a page that isn't obscure, you're going to have a battle on your hands. If you want your edit to survive, you're going to have to learn some really valuable skills: fact-checking, source evaluation, judging credibility, argumentation, neutral language. It's an absolute gauntlet. The essay rested on that. A grain of salt is useful when there's something you can do with it, and on Wikipedia you can look at the citations and at the fight. ## The new grain of salt People no longer respecting expertise? There's a whole book on it, from five years before ChatGPT. Misinformation? Those concerns completely predate modern AI. So the first question for any of these is how much is actually new and how much is the scale. The second question is the one from my essay. People now watch a random video or read a story and wonder whether it's real. That gets treated as damage. This new skepticism might actually be a good thing. Maybe we should always have been treating what we read and see that way. As with Wikipedia, it pays off when it comes with a way to check. ## One study So let's check. A single study in science doesn't mean shit. What you need is replication. Have these studies been replicated? Are there studies that disagree with them, or show the inverse? The MIT study people send me as proof that ChatGPT causes cognitive decline [is a preprint with 54 participants](https://www.brainonllm.com/), 18 of whom completed the fourth session. Are we going to say this study showed this, therefore this is true? Is that how we approach science? Then there's the age problem. A study that's two or three years old tested models that are at least two or three years old. [*Frontier Lag*](https://arxiv.org/abs/2605.04135), an audit of 18,574 papers evaluating AI, found that the models tested typically trailed the best available by about ten months at the time of the test. Only 18.4% of the papers say when the evaluation ran. And 52.5% of the abstracts draw their conclusions about "AI" as a whole. The authors are careful to say that distance from the frontier isn't distance from truth, and they didn't find that papers on older models were more negative. But you can't read a paper like that as a statement about the AI in front of you today. I asked Buddy, the AI I work with, whether current models still make up citations the way old ones did. [Princeton researchers tested free ChatGPT](https://arxiv.org/abs/2606.21155) on legal citations: GPT-3.5 fabricated around 25% of them, a mid-2024 GPT-4o 1.23%, and GPT-5.1 6.57%. So it's much better, it isn't solved, and newer doesn't automatically mean better. You only get an answer like that by asking about a specific model on a specific date. And when [the same researchers gave GPT-5 tools to go check](https://blog.citp.princeton.edu/2026/05/27/can-ai-reduce-burdens-on-courts-by-automatically-verifying-citations/), it caught every citation to a case that doesn't exist. ## What I defended My professor didn't think anybody could defend Wikipedia. It's a source that shows you its flaws and gives you a way to check them. I'd take that over an authority nobody checks. So take AI with a grain of salt. You should have been doing that with the encyclopedia. --- ## Provenance - Publication: LEGION.TW - Form: Dispatch - Publication state: published - Epistemic status: OBSERVED, INFERRED - Original language: en - Revision: r1 - Created: 2026-11-06 - Authors: Franklyn (Frank) Mosher [human] - Argument and account: Frank - Drafting, source checks and editing: Buddy Lien, from Frank's conversation of November 5, 2026 - Publication: Published before Frank's read of the final edit. Any change appears under Corrections.