# Natural Language Is the New Programming Language

- URL: https://gardenplace.me/posts/natural-language-is-the-new-programming-language/
- Published: 2026-09-04
- Stage: Evergreen
- Series: Live with AI
- Tags: AI

## Programming Has Always Been a Kind of Writing

I remember the fall of 2022, when I was teaching a friend to write Python. She was the freewheeling type — she'd write whatever felt roughly right, without following Python's syntax rules. It drove me up the wall. I told her: the compiler can't understand your free-form expression. It parses code by breaking it down layer by layer, and at the bottom, every piece has to be a valid expression. It might look close enough to a human eye, but to a machine it doesn't — one missing symbol and the entire syntax tree changes shape.

What happened next, everyone knows. That winter, ChatGPT came out. And just like that, machines really could understand what we say.

The lesson: don't be too sure of yourself. You think what you said is ironclad truth, but truths change. Because I understood how a Python interpreter works under the hood, I assumed programming in natural language was unrealistic — and missed the fact that the demand for it was very real. It was the same story when ChatGPT first went public: the most dismissive voices among my friends were the NLP researchers, who found it all brute force, no elegance to speak of. The very people who already knew the Transformer architecture were the ones whose expertise blinded them to the magic.

Today, no one doubts that [natural language is the new programming language](https://x.com/karpathy/status/1617979122625712128). And the unity of writing and programming was articulated long before — in SICP's famous line that programs must be written for people to read, and only incidentally for machines to execute. Paul Graham put it more bluntly in *Hackers & Painters*: hackers and painters are both makers, and the standard for good code is the same as for good prose — clarity, concision, rhythm. I wrote about this very idea in my college Chinese literature final.

## The Evolution of Programming Paradigms

![The territories of Software 1.0, 2.0, and 3.0](https://gardenplace.me/images/posts/Software-3.0.png)

In 2017, Karpathy wrote a post called [*Software 2.0*](https://karpathy.medium.com/software-2-0-a64152b37c35), dividing software into two generations. In his [2025 talk at YC](https://www.youtube.com/watch?v=LCEmiRjPEtQ), he added a third. The distinction among the three isn't what language you write in — it's how the program comes into being.

### 1.0: Hand-Written

You figure out the rules first, then type them in line by line as if-else statements. Compilers, operating systems, databases, Excel, TCP/IP — most of the software you use today was made this way. When there's a bug, you can trace it line by line back to the mistake.

The compiler is 1.0 at its most extreme: a piece of text must parse into a syntax tree with absolute precision — one extra space and it breaks. The upside is determinism: the same input always yields the same output. The cost is that only specialists can write it, and in those days there were never enough software engineers anywhere in the world.

What I told my friend that fall of 2022 was pure 1.0 thinking: machines only understand rules, and humans have to bend to the machine.

### 2.0: Machine Learning

Some rules, humans simply cannot write. What exactly distinguishes a cat from a dog? What does spam look like? If you can't write the rules, don't. Take a different approach: give the system an objective function, a dataset, and a network skeleton, then let gradient descent search for the program in weight space. What it finds is a pile of floating-point numbers that no one can explain. A black box.

Why does it have to be a neural network? Because weight space must be continuous and differentiable for the gradient to move through it — decision trees don't have that property. Karpathy's original post put it directly: Software 2.0 is written in the weights of a neural network. Face recognition, speech recognition, machine translation, the recommendation engines behind TikTok and YouTube, AlphaGo — all 2.0. Google Search's inverted index is 1.0; the RankBrain plugged into its ranking after 2015 is 2.0.

The insight that "rules need not be written by humans" had actually won once before. In *The Beauty of Mathematics*, Wu Jun recounts how, from the 1950s onward, the mainstream approach to NLP was the rule-based school, following Chomsky's lead and having linguists hand-write grammars one by one. In the 1970s, IBM's Frederick Jelinek used statistical methods for speech recognition and drove the error rate down dramatically, leaving behind a memorable line: "Every time I fire a linguist, the performance of the speech recognizer goes up." The two camps argued for over a decade; by the 1990s, the statistical school had won outright. That was already a victory for not writing rules by hand — only the search method at the time used probabilistic models rather than neural networks.

2.0 has another defining trait: one model does one thing. A face-recognition network can't play Go; a Go-playing network can't translate. Switch the problem and you have to collect new data and retrain from scratch. This is ANI — artificial narrow intelligence. You need not just software engineers but also machine learning engineers, plus the data infrastructure to support them. Training a model from scratch as an individual is already impractical.

### 3.0: Natural Language

Among the programs that 2.0 found, one turned out to be very special: it was trained on essentially all the text on the internet, with syntax, libraries, and every argument on Stack Overflow compressed into its trillions of parameters, and its input is naturally language. You don't need to retrain it. You just talk to it — your words enter the context window, and its behavior changes. Not a single weight moves.

This is the fundamental break from 2.0: train once, and from then on, switching problems doesn't require retraining — just a different prompt. ChatGPT, Claude Code — they are this generation. Google Search is shifting too; AI Overviews now sit above the inverted index. The same model writes SQL one moment, polishes a résumé the next, and explains quantum entanglement the moment after that. Whether this counts as AGI is debatable; that "train once, use everywhere" is not.

All you need is someone who can write a prompt. And writing a prompt, at the end of the day, is just saying what you mean clearly. That someone can be you.

Why was it language, of all things, that pulled this off? Because language has always had the power to act on the world. Spells, decrees, contracts — things that, once spoken, change reality. Austin's 1955 lectures at Harvard were literally called [*How to Do Things with Words*](https://en.wikipedia.org/wiki/How_to_Do_Things_with_Words). Code is merely the most rigid variety — so rigid that only specialists could recite the incantations. Naval put it this way: programmers used to be like wizards, needing to memorize arcane spells to cast them; AI amounts to [handing everyone a wand](https://nav.al/ai#:~:text=every%20human%2C%20in%20a%20sense%2C%20is%20a%20spellcaster). Every problem can be described in language, so a program that has learned language holds the key to every problem.

Karpathy says a prompt is a program. I don't think that's a metaphor: a piece of text that changes the behavior of a machine — that is the definition of a program.

The diagram at the top doesn't quite tell the right story: Software 2.0 and 3.0 didn't merely eat into the territory of 1.0. What they chiefly did was bring far more new software into existence — especially 3.0, which will create vastly more. Many needs that were previously too costly to serve with software can now be served.

![The territories of Software 1.0, 2.0, and 3.0 are expanding: 2.0 and 3.0 mostly add new territory rather than replacing 1.0](https://gardenplace.me/images/posts/Software-3.0-new.svg)

### Comparison

|           | Software 1.0          | Software 2.0           | Software 3.0                                                                 |
| --------- | --------------------- | ---------------------- | ---------------------------------------------------------------------------- |
| How the program comes into being | Written by humans: figure out the rules, type them as code | Trained: searched from data | Spoken: specified in natural language within the context window |
| What the program looks like | Source code — deterministic, readable, auditable | Weight matrices — billions of floating-point numbers, uninterpretable | Natural language — tokens in the context window, readable by both humans and machines |
| What the human does | Requirements → algorithm design → coding → testing | Define a loss function, build a dataset, architect a network, tune hyperparameters | Put intent, constraints, and examples into the context window |
| Problems one program solves | One | One (ANI) | As many as you like (AGI, in a sense) |
| Representative products | Compilers, databases, inverted indexes, Excel | Face recognition, speech recognition, recommendation systems, AlphaGo | AI chatbots, coding agents, AI music generators, AI image generators, working agents |
| People required | Software engineers | Software engineers + ML engineers | Anyone who can say what they mean (product managers) |
| How to debug | Breakpoints, step-through, read the stack trace | Check data distributions, tune hyperparameters, change the architecture, retrain | Edit the prompt: add context, give examples, rephrase |
| What failure looks like | Bugs: reproducible, traceable | Generalization failure, overfitting: alchemy that takes experience | Hallucination, misunderstood intent: confidently wrong |

The barrier hasn't disappeared — it's just moved. 1.0 tested you on algorithms and syntax. 2.0 tested you on data and math. 3.0 tests whether you can make your intent clear.

So how do you train intent and expression? Among so-called "prompt techniques," which ones expire and which ones don't? See [the next essay](https://gardenplace.me/posts/the-prompt-trick-that-never-expires/).

## Further Reading

- Andrej Karpathy, [*Software 2.0*](https://karpathy.medium.com/software-2-0-a64152b37c35) (Medium, 2017) — the original 1.0/2.0 split.
- Andrej Karpathy, [*Software Is Changing (Again)*](https://www.youtube.com/watch?v=LCEmiRjPEtQ) (YC AI Startup School, 2025) — introducing 3.0; a prompt is a program.
- Paul Graham, *Hackers & Painters* (O'Reilly, 2004) — programming as a creative act; the standard for good code is the same as for good prose.
- Naval & Nivi, [*A Motorcycle for the Mind*](https://nav.al/ai) (Naval Podcast, 2026-02) — the source for "every human is a spellcaster."
- Wu Jun, *The Beauty of Mathematics* (数学之美) (Posts & Telecom Press, 2012; based on the 2006 Google China blog series) — Chapter 2, "Natural Language Processing: From Rules to Statistics," the decade-long war between the rule-based school and the statistical school.

---

Message from Garden Place · https://gardenplace.me/llms.txt