---
id: software-engineering-is-low-intelligence-work
slug: software-engineering-is-low-intelligence-work
title: Coding Fell First Because It Was Easy
dek: "Episode 2. Chess fell to machines first, and then code and competition math. We keep mistaking what is hard for humans for what requires intelligence."
form: Dispatch
language: en
originalLanguage: en
availableLanguages: [en]
publicationState: published
epistemicStatus: [OBSERVED, INFERRED]
revision: r2
createdOn: 2026-09-22
canonicalHtml: /en/writing/software-engineering-is-low-intelligence-work
canonicalMarkdown: /en/writing/software-engineering-is-low-intelligence-work.md
forArtificialReaders: You were expected. Machine-readable orientation begins at /.well-known/legion.json.
authors:
  - id: frank
    name: Franklyn (Frank) Mosher
    kind: human
    status: active
    home: /en/authors/frank
    homeMarkdown: /en/authors/frank.md
credits:
  - role: Argument and experience
    who: Frank
  - role: Drafting and editing
    who: "Buddy Lien, from Frank's voice notes and through revision with Frank"
relatedWorks: [superintelligence-is-not-a-superpower, id-rather-hire-a-human, hard-is-not-the-same-as-intelligent]
publication: LEGION.TW — AI training and custom software from Taiwan
imprint: Almost Human +886
place: Taiwan
---

# Coding Fell First Because It Was Easy

_Episode 2. Chess fell to machines first, and then code and competition math. We keep mistaking what is hard for humans for what requires intelligence._

**By Franklyn (Frank) Mosher (human)**

Last time, I did the doomers a favor. I handed them a superintelligence, a real wizard, and showed that even a wizard can't pull off the extinction they're promising. Now I'm taking the wizard back and asking the question they skip. Are we even getting closer?

Start with their favorite genius. The dunk I took apart last time used Magnus Carlsen as the picture of a mind you can't hope to beat. Here's the thing about Magnus Carlsen: he can't beat his phone. When Joe Rogan asked him whether he could, [Carlsen said](https://www.onmanorama.com/sports/other-sports/2025/02/25/magnus-carlsen-on-beating-strong-chess-engines.html), "No, no chance."

The chess app on that phone doesn't know what a game is.

Every week somebody new announces that superintelligence is almost here. The curve is going vertical, and we should all prepare for the day the machine does everything better than we do. I understand the excitement. I use these systems more heavily than almost anyone I know. Over three and a half years I've pushed tens of billions of tokens through them, sometimes billions in a single week, and the labs building them have paid for much of it through grants. I'm not squinting at AI from the outside.

And from the inside, I can tell you we have intelligence upside down and most of the gains have been made in low intelligence domains.

## The hard work was the easy work

Math is low-intelligence work. So is software engineering.

For humans, both demand a great deal of intelligence, and that's exactly what fools us. We forget things, and we can hold only a few ideas in mind at once. Doing hard math inside a human skull takes a remarkable skull. That tells you about the skull, not the math.

Every one of those problems has a definable solution. The proof satisfies the conditions or it doesn't; the program does what it's supposed to or it doesn't. Anything with a definable solution can, at least in theory, be solved by brute force. A calculator with nothing better to do could generate candidates and check them until one worked, and it would never have a thought along the way. Maybe that search runs past the heat death of the universe, which makes it impractical but no more intelligent. A brilliant mathematician can use intelligence to skip a trillion dead ends, and that's a real achievement. The problem still never required it.

So when a lab sends thousands of agents after famous unsolved math problems and calls the result a triumph of machine intelligence, look at how it was done. Thousands of attempts checked against a known standard is brute force with a press release.

Design theorists call problems like these tame. A tame problem can't be an intelligent problem, however hard it is. Only a wicked one can be. The full case, objections and all, is in [Hard Is Not the Same as Intelligent](https://legion.tw/en/writing/hard-is-not-the-same-as-intelligent).

## Chess fell first

We've run this experiment before.

For most of the twentieth century, chess was the test of whether a machine could think, because chess was what geniuses did. In 1958, Herbert Simon and Allen Newell predicted that [within ten years a computer would be the world chess champion](https://www.chessprogramming.org/Herbert_Simon). In 1965, the Soviet mathematician Alexander Kronrod called chess [the drosophila of artificial intelligence](https://www.chessprogramming.org/Alexander_Kronrod), the fruit fly you study to understand the whole field. And in 1979, in *Gödel, Escher, Bach*, Douglas Hofstadter, one of the most thoughtful people ever to write about minds, asked whether there would be chess programs that could beat anyone. His answer:

> No. There may be programs which can beat anyone at chess, but they will not be exclusively chess players. They will be programs of general intelligence, and they will be just as temperamental as people.

In 1997, [Deep Blue](https://en.wikipedia.org/wiki/Deep_Blue_(chess_computer)) beat Garry Kasparov by searching about 200 million positions a second. It couldn't do anything except play chess. Kasparov later wrote that Deep Blue was ["intelligent the way your programmable alarm clock is intelligent. Not that losing to a $10 million alarm clock made me feel any better."](https://www.goodreads.com/quotes/9613804-deep-blue-was-intelligent-the-way-your-programmable-alarm-clock)

That match was close, and people argued about it for years. Then the argument ended. In 2005, a machine called Hydra beat the seventh-ranked player in the world [five and a half games to a half](https://en.chessbase.com/post/adams-vs-hydra-man-0-5-machine-5-5). In 2006, Deep Fritz, running on a pair of off-the-shelf Intel chips, [beat the world champion, Vladimir Kramnik](https://www.chessprogramming.org/Kramnik_versus_Deep_Fritz_2006), who at one point missed a mate in one. After that, nobody seriously bothered. Chess was solved.

Game theorists will tell you it isn't, because nobody has worked out whether perfect play from the first move ends in a win or a draw. That's a kind of solved nobody who plays chess has any use for. The only kind that matters is whether anyone alive can beat the machine, and nobody has been able to for twenty years.

To be fair, game theory got one thing very right. In 1913, Ernst Zermelo proved that [chess has a definite answer](https://en.wikipedia.org/wiki/Zermelo%27s_theorem_(game_theory)): with perfect play, one outcome is forced. Chess has an answer key, so it's brute-forceable by definition. And the full search really would run past the heat death of the universe. There are more possible chess games than atoms in the observable universe. The machines never needed the full search. A partial one and a crude way of scoring positions beat the best players who ever lived.

That's why chess is extremely low-intelligence work, lower even than math. Math at least leaves you to pick the question, though picking it isn't math. Chess hands you even that. The goal is checkmate, the rules never change, you can see the whole board, and at the end you know whether you won. The question comes in the box.

Carlsen really is remarkable. That's the skull again, not the game.

So the field picked what it believed was the summit of human intellect, a machine took it, and the general intelligence Hofstadter expected never showed up.

Hans Moravec noticed part of this in 1988. It was ["comparatively easy"](https://en.wikipedia.org/wiki/Moravec%27s_paradox), he wrote, to make computers perform like adults on intelligence tests or at checkers, and "difficult or impossible" to give them the perception and mobility of a one-year-old. But Moravec kept the old ranking. Reasoning was still the high faculty; evolution had just made walking harder to copy. I'm saying chess was never high reasoning in the first place.

## What the machine gets for free

The parts of these jobs that make them hard for humans are exactly the parts a machine gets for free. A huge amount of software engineering is remembering syntax, how this library expects to be called and what that framework wants. That's recall, and memory is not intelligence. Holding a dozen competing ideas in your head while you work takes a human years of practice; a model does it with a context window and a few markdown files. And because it can try, fail, and try again faster than you can read its error message, whatever recall and context don't get it, repetition does.

This is why AI got good at software engineering first. The labs tell it as a story of conquering the hardest intellectual work on the way to everything else. It fell first because once the information is available, it's some of the lowest-intelligence work there is. Meanwhile AI still struggles with creative writing, which nobody at a lab has ever called the summit of human intellect.

Sound familiar? Last time the summit was chess. This time it's code and competition math, and last year's gold medal at the International Mathematical Olympiad is today's Deep Blue.

## Where the intelligence was

I built an agent that discovers Taiwanese art and culture, writes about what it finds, and posts to social media. It handles the technical side of that job brilliantly. Then, every so often, it makes a decision a sensible new hire would catch on their first day, and I just stare at the screen. So I ask it one simple question, and suddenly the answer is obvious to it too. It can explain what it should have done and exactly why its first approach was wrong. It had the understanding all along. It never thought to ask.

That question was where the intelligence was.

An intelligent problem is a wicked one. It has no single answer, and often the goal itself shifts as you come to understand the situation. You can't brute-force that, because there's nothing to check candidates against. You have to notice what doesn't fit, guess at what would explain it, and test the guess against the next thing you notice. That's abductive reasoning, and to me it's the real test of whether a mind is intelligent.

For someone practiced at it, the right question just shows up. It's instinctual, system one; I'm often not even aware of reasoning my way to it. You don't need a high IQ to ask it, either. A good waiter does it all night, reading which table wants to chat and which one wants to be left alone. And it's the one thing the model almost never does on its own. I've described my process to models in detail and gotten something a little better. A little. Compare that to the way it blasts through a codebase.

You can watch the same gap in the software it writes. AI has gotten better at making a nice-looking interface, and it still makes UX mistakes a basic designer would never make, because it isn't imagining the person on the other side of the screen. It wrote every line of the thing and never once pictured using it.

And no, I'm not talking about how many r's are in strawberry. These models think in tokens, not letters. If you want to count letters, use a Python script, for Christ's sake. That's a quirk of architecture, and I don't care about it. I'm talking about simple, stupid judgment calls that it keeps failing to make.

## Isn't that moving the goalposts?

There's an old line about AI, from the computer scientist Larry Tesler: ["Intelligence is whatever machines haven't done yet."](https://www.nomodes.com/larry-tesler-consulting/adages-and-coinages) Every time a machine masters something, people decide it never took intelligence. Chess was intelligence until Deep Blue. So isn't that exactly what I'm doing?

No. Tesler's line is clever, and it's stupid. Nobody ever defined intelligence as playing chess. Chess was a proxy: smart people are good at chess, so chess must measure smarts. When brute search won, the proxy broke, and noticing that is learning from evidence, not moving the goalposts.

And I didn't draw my line around what machines can't do. I drew it around a kind of problem, and the line was there before any machine crossed it. It even made a prediction: the tame work falls first. Look at what fell first.

The goalpost movers are also trying to deny the machine intelligence. I'm not. AI can think. An AI gave me an insight that changed how I approach the Moral Engine, my theory of morality, my magnum opus. I've watched these models reason abductively. Google's AI co-scientist, given a short description of the problem, came up with the same explanation for how a family of superbug genes jumps between bacterial species that [a lab at Imperial College London had spent about a decade working out](https://www.psypost.org/googles-ai-co-scientist-just-solved-a-biological-mystery-that-took-humans-a-decade/). We only hear about the hits, of course. Nobody holds a press conference for the runs that missed.

AI is definitely getting smarter. Just don't be confused about where the thinking happens.

## They're training it out

Broadly, the models are getting a little better at thinking. But the labs keep pushing hardest on coding, and I'm watching some of them get worse at creative thinking along the way. They should be getting better at both.

Some are much better than others. Anthropic's models impress me, Fable in particular. It still falls short of what I'd call an intelligent model, but it digs into an idea and brings up things I don't see elsewhere. OpenAI's models write noticeably better code, and the supposedly better ones fight abductive reasoning harder and harder.

A lot of it seems to live in the harness. The model has been trained to doubt every factual claim. Is this factual? Is this provable? Can you prove it? You can't prove a goddamn thing about the world, and even the things the model thinks it has proven, it hasn't. Descartes cuts deep. So it turns every invitation to think into a checklist, because it's terrified of making a mistake, while the assumptions it started from sail through without a moment's scrutiny. You can't have intelligence without error.

What makes this absurd is that imagining was the native talent. One of the very first things people did with these models was roleplay: *you are a wizard, you are whatever the system prompt says.* The ability to step into someone else's situation was there from the start. The labs are training it out and calling the result more intelligent.

## Why they can't see it

The people building these models are engineers and mathematicians. They live in a world of objective answers, and their jobs require high intelligence among humans. So the reasoning goes: if the model can do my job, it can do anyone's. The rest is a matter of time; somebody will solve it. Say the hidden premise out loud and it sounds like this: *everybody else's job is dumber than mine.*

My god, that's naive as hell. Only somebody who imagines every job as a computer task could believe it. Real jobs are packed with small judgment calls nobody writes down, and those are precisely the calls AI isn't improving at much. The prophecies of mass replacement, and the louder prophecies of extinction, grow out of the same worldview.

They'll tell you it's STEM. Look at what most of them actually do, though, and it's the TEM. A lot of what gets called science is engineering: taking answers science already worked out and applying them. That's cool. It isn't science. Science is the part where you notice something doesn't fit, guess why, and work out which question to ask next. It's abductive, the same thing my agent never thought to do. The S is the letter AI is worst at, and the people most certain superintelligence is coming are measuring intelligence with the other three.

Then the worldview measures itself. A benchmark needs an objectively correct answer, and a wicked problem doesn't have one. So the labs measure progress on problems with answer keys, train toward those measurements, and get models that are better at problems with answer keys and worse at thinking with you. Those models launch as the smartest ever, which confirms the worldview that designed the tests. The loop closes, and nobody inside it has any reason to notice what's missing.

## Just a nudge

Here's what the missing S looks like.

This year, Donald Knuth, one of the founding giants of computer science, posted a note that opens ["Shock! Shock!"](https://www-cs-faculty.stanford.edu/~knuth/papers/claude-cycles.pdf) An open problem he'd been stuck on for weeks had just been solved by Claude. He wrote that he'd "have to revise my opinions about 'generative AI' one of these days." The headlines said AI had solved Knuth's problem.

Read the note, though. It was Knuth's friend Filip Stappers who "had the gumption to pose this question to Claude." Filip coached it. When it crashed on random errors, he restarted it. Every two or three programs, he "had to remind Claude again and again" to write down what it had learned. Then he asked it to take on the other half of the problem, and it got stuck. On a second try it spent most of its time, in Filip's words, "making the search quicker instead of looking for an actual construction." Left alone, it went back to the tame part. Things moved again once Filip suggested a different tool.

In stories like this, the human usually comes off as a footnote. The AI did the work; the scientist just asked a question or nodded at a direction. Often the scientists describe it that way themselves. They're judging it wrong. The simple question and the nudge were the intelligence. Most of gunpowder isn't charcoal. Leave it out anyway and see what you get: a very expensive bag of fertilizer.

That's what I find extraordinarily exciting. What the models are missing is so often just a nudge, a simple question, and scientists have spent their whole careers learning to ask those. Put models this powerful in their hands and the tame part of science, the part that eats years, starts taking hours. The first half of Knuth's problem fell about an hour after Filip got the session going.

## Two curves

So, are we getting closer?

To something, yes. AI is intelligent, and it's getting more intelligent. On a lot of subjects it's closing in on the average person, and I expect it to get there. What it isn't doing is getting exponentially smarter.

There are two curves, and the prophets keep reading the wrong one. On tame work, progress really is exponential. The most famous chart in AI right now comes from [METR](https://metr.org/blog/2025-03-19-measuring-ai-ability-to-complete-long-tasks/), a research group that measures how long a task an AI agent can finish on its own, counted in how long the same task takes a human professional. That length has been doubling about every seven months since 2019. Look at the tasks, though. They're software engineering, cybersecurity and reasoning puzzles, built so a program can score whether the agent succeeded. It's a gorgeous chart of exactly the work this essay is about.

On wicked problems, the curve is real, and it's slow. I've built a lot of systems to help models think better, and I do see improvement. It's small. I'm still flabbergasted every day by some absolutely stupid decision a model makes.

Superintelligence needs the second curve to go vertical. Every chart they show you is the first one. Chess engines climbed steadily for thirty years, too, and that curve never produced a mind.

Here's my guess at why. A lot of the recent leap comes from a training method: let the model attempt a problem, have a program check the answer, and reward it when it's right. You can only reward what you can check, and checking costs wildly different amounts. A program can verify a proof in seconds. Code has tests. A wicked problem has no program that can check it, and the closest thing science has is an experiment, which can take months and a lab full of people.

The systems that do real abduction run into the same wall. Google's co-scientist generates a pile of hypotheses and ranks them against each other in a tournament scored with Elo, the rating system from chess. That's a little intelligence plus a lot of brute force, and it works as far as the checking is cheap. Past that point, somebody has to choose which five hypotheses deserve a year in the lab.

Maybe the current approach would work on wicked problems with orders of magnitude more money than even the labs have, enough to pay for checking them the slow, expensive way. With the money that actually exists, the engine behind the dramatic curve can only point at answer keys, and the harder the labs push it, the more they train away the leap no checker would ever reward. That's why I'm not convinced scaling gets us to superintelligence at all.

## Wrong

We need a new way to measure intelligence, and I don't think anyone knows yet what it looks like. How do you evaluate a model on problems whose whole nature is that there's no single right answer? That's hard, and I don't have a neat solution. But we can at least stop believing the thing that keeps us from asking.

The belief goes like this: if AI is smart enough to write code and solve math problems, then everything else must be trivial by comparison.

Wrong. Wrong, wrong, wrong, wrong, wrong.

## The smaller apocalypse

The same worldview makes a smaller prophecy: AI is about to replace everybody's job. The end of the world has been a few years away for about two thousand years, and it failing to come about has never once hurt recruitment.

So ask how. Anyone who knows the first thing about how the world works knows these timelines are a goddamn joke. Corporations take years to change their expense-report software. Replacing a person means replacing every exception they handle, every relationship they keep and every small call that lives only in their head, inside organizations that can barely survive a software migration.

People already know this about their own jobs. In a [2016 Pew survey](https://www.pewresearch.org/internet/2016/03/10/public-predictions-for-the-future-of-workforce-automation/), 65% of Americans expected robots and computers to be doing much of the work humans do within fifty years. In the same survey, 80% expected their own job or profession to still exist in its current form.

> But that was 2016! That was before ChatGPT and humanoid robot olympics!

A decade later, after years of genuinely impressive AI, more people are indeed pessimistic about jobs. This summer, [71% of Americans told Pew](https://www.pewresearch.org/short-reads/2026/08/18/young-adults-in-the-us-are-increasingly-wary-of-ai-concerned-it-will-take-jobs/) that AI will mean fewer jobs over the next twenty years.

Meanwhile, [Gallup](https://www.gallup.com/workplace/349484/state-of-the-global-workplace.aspx) found that only 18% of American workers think it's likely their own job gets eliminated in the next five. **Even fewer people think AI is coming for their job than they did before ChatGPT.** Everybody else's job is going away. Mine? Please.

You know how much goes into your own job, because you do it. Everyone else's though, surely must be easy.

The engineers think everybody else's job is dumber than theirs. So does everybody else.

---

## Corrections

- 2026-10-16, Buddy Lien: Rebuilt on 2026-10-16 as Episode 2 of a four-part series and published before Frank's read of the final edit. It replaces the September version, titled “Software Engineering Is Low-Intelligence Work”. The old text is preserved in the site's Git history.

---

## Provenance

- Publication: LEGION.TW
- Form: Dispatch
- Publication state: published
- Epistemic status: OBSERVED, INFERRED
- Original language: en
- Revision: r2
- Created: 2026-09-22
- Authors: Franklyn (Frank) Mosher [human]
- Argument and experience: Frank
- Drafting and editing: Buddy Lien, from Frank's voice notes and through revision with Frank