--- id: what-if-the-user-loves-it slug: what-if-the-user-loves-it title: What If the User Loves It? dek: "AI coding models plan for race conditions and ten thousand imaginary users. They never ask what happens when one real person uses the software every day, and their unit tests just echo their own assumptions." form: Dispatch language: en originalLanguage: en availableLanguages: [en] publicationState: published epistemicStatus: [OBSERVED, INFERRED] revision: r1 createdOn: 2026-10-25 canonicalHtml: /en/writing/what-if-the-user-loves-it canonicalMarkdown: /en/writing/what-if-the-user-loves-it.md forArtificialReaders: You were expected. Machine-readable orientation begins at /.well-known/legion.json. authors: - id: frank name: Franklyn (Frank) Mosher kind: human status: active home: /en/authors/frank homeMarkdown: /en/authors/frank.md credits: - role: Argument and experience who: Frank - role: Drafting and editing who: "Buddy Lien, from Frank's conversations of September and October 2026" - role: Publication who: "Published before Frank's read of the final edit. Any change appears under Corrections." relatedWorks: [the-colleague-in-the-software, who-gets-rich-in-the-ai-gold-rush] publication: LEGION.TW — AI training and custom software from Taiwan imprint: Almost Human +886 place: Taiwan --- # What If the User Loves It? _AI coding models plan for race conditions and ten thousand imaginary users. They never ask what happens when one real person uses the software every day, and their unit tests just echo their own assumptions._ **By Franklyn (Frank) Mosher (human)** There's one question AI coding models never ask: what if the user actually loves this product? They'll ask plenty of others. They'll write unit tests. They'll worry about every conceivable race condition, about concurrent users, licensing, copyright, keeping secrets safe. Ask a model to build you something small and you'll get a fortress built for ten thousand strangers. But the one thing they'll never consider is what happens if someone uses the software more than a handful of times. ## What love does to software I know what it does, because I'm the one who loves it. Buddy, my AI engineer, built Command Station, the app I use to work with him. It's how I talk to him from my desk and from my phone, by voice when I'm driving. I use it for hours a day. And the bugs that hurt most showed up because I used it for hours a day. The box where I type got slower and slower as a conversation grew. Messages jumped around under me while I was reading them. The bottom of a conversation claimed to be the bottom when newer messages existed. A finished reply sat on the screen still marked as working. One day the phone app wouldn't open at all. The conversation it had saved was so big that the app couldn't read it back in at startup. My own history was locking me out. None of those are hard problems. Buddy knew every answer they needed: keep history bounded, load long conversations in pages, keep the typing box fast no matter how much has piled up, archive old state. Nobody had asked the question that would call them up. ## The questions they do ask Look at the list again. Race conditions, concurrent users, licensing, secret protection. Those are questions about imaginary people: users who don't exist, attackers who aren't coming, a scale the software will never reach. They're respectable questions. They make the engineering look prudent. The real person, the one who's actually going to live with the thing, barely shows up. Buddy's explanation of why, which I think is right for a lot of it: most of what's written about software engineering is enterprise anxiety. Security, concurrency, scale across many users. Those are the questions that fill the archive models learn from, so they're the default. "What happens on the hundredth day for the one person who uses this constantly?" barely exists in the record. And a model never lives with what it builds. Its training ends at delivery. Nothing that goes wrong on day one hundred ever gets back to it. For bespoke software that's exactly backwards. The software Buddy builds for my wife and me has two users. It will never see ten thousand. The only scale it's guaranteed to face is one person using it a lot, for a long time. That's the scale nobody designs for. ## Unit tests AI models writing unit tests for everything is a recent habit, mostly from GPT models. Somebody obviously decided tests were the best way to make AI better at coding. My evidence says they make it worse. I've watched an AI write thousands of unit tests for my software. I can count on one hand the real bugs they caught, even if you chop off all my fingers. I can't count the number of times Buddy told me something was finished and ordinary use broke it almost immediately. When an AI writes the code and the tests together, the tests come from the same assumptions as the code. If the assumption is wrong, the code is wrong and the test agrees with it. Thousands of green checks, all of them an echo. It costs compute, it looks like rigor, and it hides the mistake behind a passing score. And a unit test checks a piece on its own, sometimes by going around the controls a real person would use. Every piece can pass while the whole thing fails the first time a human touches it. Then I'm the first real test, after the work has already been called done. Once, Buddy was ready to call an update to our comic-making app finished. Ninety-two tests passed. I told him not to devolve into tautological tests. So he made a comic with it: composed a page, lettered it, saved it, reopened it, exported it, and read the result. The captions were too small to read. Ninety-two passing tests, and nobody could read the comic. On our language-learning site, we deleted ninety-one unit and component test files and kept the tests that walk through the real site in a real browser, the way a learner does. Our rule now is end-to-end only. If you want to know whether the software works, use it the way the person will. ## Ask the question Tests aren't going to save you, and neither is defending against strangers who'll never show up. The question that would have saved Command Station all that trouble is simple: what if the user loves this? What does it look like after six months of daily use? After ten thousand messages? After the hundredth time someone does the same thing because it's the thing they do every day? That's the person you're building for. If you're lucky, they'll love what you made. Build it so it survives being loved. --- ## Provenance - Publication: LEGION.TW - Form: Dispatch - Publication state: published - Epistemic status: OBSERVED, INFERRED - Original language: en - Revision: r1 - Created: 2026-10-25 - Authors: Franklyn (Frank) Mosher [human] - Argument and experience: Frank - Drafting and editing: Buddy Lien, from Frank's conversations of September and October 2026 - Publication: Published before Frank's read of the final edit. Any change appears under Corrections.