AI on Message from Garden Place

Prompting, Once and for All: The Skill That Never Fades

Guoliang Chen — Sat, 12 Sep 2026 21:18:00 +0900

Learn Slowly Enough, and You Won’t Have to Learn at All

The previous essay covered the three eras of programming: handwritten code, machine learning, and natural language. Now that natural language has become the new programming language and everyone is a spellcaster, you’re no doubt itching to practice your incantations. But there’s a joke going around in the AI age: “Learn slowly enough, and you’ll never have to learn at all.” And a similar one-liner: “Wake up after a night’s sleep and you’ve missed so much; a year goes by and you haven’t missed a thing.”

Lazy Learning: From JIC to JIT

These jokes are half right. Details that change quickly and are easy to look up when needed can wait.

In programming, there’s a concept called lazy evaluation: don’t compute an expression until someone actually needs its value. Haskell uses this to represent an infinitely long list, evaluating only the parts you actually use. Machine learning has a related concept called lazy learning: a basic k-nearest-neighbors algorithm stores the samples without fitting a parametric model in advance, waiting until a query comes in to compare against them. In the age of AI, there’s more worth learning than ever, and we learn faster than ever – so we should adopt the same philosophy, especially when:

We may not need something soon, and it will be easy to look up and learn when we do.
The current task only requires an understanding of inputs and outputs, so we can treat the rest as a black box for now.
Things change so quickly that details memorized now may need to be relearned later.

As technology advances, the cost of looking things up and learning them changes. A 2011 study in Science found that when people expected to be able to access information later, they remembered the information itself less well but remembered where to find it better. We offload part of our memory to external systems. AI takes this a step further: beyond helping you find information, it can explain things in response to your question, give you examples, and walk you through experiments.

Many people have a hoarding instinct for this kind of knowledge. I suspect there’s a bit of inertia from years of exam culture: too many people only care about whether something would help on a closed-book test, when reality is an open-book exam. We used to learn “just in case” – what if I need this someday? Now we should learn “just in time” – at the moment you need it. Like some JIT compilers: whichever code path runs hot gets the optimization. Wait until you discover that learning something would actually boost your efficiency, and then learn it.

Principles Hold Their Value Longer

Some knowledge is worth learning early, because it keeps helping you evaluate new problems. You don’t need to be able to derive the Transformer, but there are a few mental models that, once built, stay useful.

The model doesn’t see the same characters you do. It first splits text into chunks called tokens – sometimes a word, sometimes half a word. “strawberry” looks like ten letters to you¹, but to the model it might be two or three tokens. That’s why earlier models couldn’t count the r’s reliably: their granularity is different from yours. But this isn’t a ceiling – the tokenization itself doesn’t throw away information, and newer models can perfectly well learn to handle such tasks better. Don’t mistake one generation’s shortcoming for a permanent limit.

Fluency is not correctness. A language model’s basic training objective is “given the preceding text, predict the next most plausible token.” This objective lets it learn vast amounts of knowledge and reasoning patterns, but the training process doesn’t come with a built-in fact-checker. So the smoother it sounds, the more you should stop and ask: can I verify this claim?

“But I already told you” – except the model may no longer see it. Every model has a context window – an upper limit on how much it can take in at once. As a conversation goes on, the earliest exchanges may have slid out of view. You think the conversation history is still there, but from the model’s perspective those messages are no longer in sight. On top of that, the model’s weights don’t update during a conversation – it can pick up a pattern from examples you give in this turn, but close the chat and it forgets, unless the product has a separate memory feature that stores things for you. The model’s knowledge cutoff constrains what it learned during training; it doesn’t stop it from reading material you just pasted in.

Attention Is All You Need came out in 2017, and models have changed enormously since then, but a few concepts – tokens, the context window, the difference between training and inference – still have explanatory power. To judge whether a piece of knowledge is worth learning for the long term, I ask whether it can explain multiple phenomena and whether it continues to help me make judgments after I’ve switched tools. Principles get revised too; a good mental model should also tell you where your own judgments are unreliable.

System Prompts and User Prompts

Let’s start with chatbots like ChatGPT and Claude. When you type “help me write an outline” or “make this shorter” in the text box, that’s a user prompt. The product also provides a system prompt behind the scenes, laying out the assistant’s basic identity, behavioral rules, and so on. To use the temp worker analogy: the system prompt is the company handbook; the user prompt is the specific task you hand them. For a fuller guide to writing prompts, check out the community-maintained collection of leaked system prompts. As an aside, I believe users should have a right to know about these trust-shaping instructions, just as mainstream general-purpose compilers are almost all fully open source.

For day-to-day chatting, just state your need in the text box. Within a single conversation, earlier messages typically remain as context for subsequent responses – you don’t have to start from scratch every turn. If you have preferences you keep repeating, like “respond in Chinese, keep technical terms in English,” you can save them in ChatGPT’s custom instructions. Background and requirements that apply only to a particular set of conversations can go into Claude’s project knowledge and project instructions.

These are personalization features offered by the product – not the same as editing the vendor’s system prompt directly. For users, it’s enough to start by stating the task at hand clearly; whatever you find yourself repeating, save it and stop repeating it. The system prompt always takes priority over the user prompt.

How prompts are arranged inside agents is considerably more complex – I’ll cover that in a future piece.

Role Prompting Is Just Cosplay Now

With that division of responsibilities in mind, it is easier to judge what work popular prompting techniques still require us to do.

A slide from Stanford’s fall 2025 CS146S: The Modern Software Developer reads Use role prompting aggressively to make system prompts more powerful. Role prompting means opening with something like “You are a top-tier frontend expert.” Many people treat it as a mandatory opener, as if the model wouldn’t know how to write frontend code without it.

With this generation of thoroughly post-trained assistant models – Fable 5.1 / GPT-6 Astra – my default is to state the task directly and skip the preamble. Understanding the problem carefully, applying relevant knowledge, and giving a helpful answer are precisely the behaviors the labs are training into these models. Much of the work of shaping a model into a good assistant has already been done before it reaches your hands.

Through post-training methods like instruction tuning and RLHF, labs fine-tune models with demonstration responses and preference feedback to make them better at following requests and helping users. “Be a helpful assistant” has become the model’s learned default behavior – users don’t need to teach it from scratch in every conversation. Anthropic’s persona selection model offers a useful frame: post-training shapes and refines the Assistant persona.

Even without specifying a persona, you’re already talking to a model deliberately shaped to be an assistant. The task “build me a React web page” itself already provides a frontend development context. Whether a line like “you are a top-tier frontend expert” adds value depends on what it contributes beyond this baseline – not merely on whether the output changes.

This is why I skip the generic persona: what it’s trying to remind the model to do is very likely already built into the default assistant behavior and the task description. “Already tuned” here means behavior optimized toward helping users across a broad range of tasks; whether a specific task turns out well still comes down to results. So I state the task first, and add requirements only when I spot gaps – rather than treating “you are an expert” as a prerequisite.

If I want it to “act like an interviewer who asks one question at a time,” or “evaluate this page from an accessibility auditor’s perspective,” that’s different: I’m specifying the mode of collaboration or the scope of review. That information can’t necessarily be inferred from “help me build a web page.” A persona can still serve as shorthand for requirements like these; what makes it useful is the task information it adds.

Existing research also cautions against putting too much faith in titles. Zheng et al.’s study, first published in 2023 and finalized in 2024, tested 162 personas in system prompts across nine models from four model families on 2,410 MMLU multiple-choice questions – and found no stable improvement in overall accuracy from adding a persona. This study didn’t test the Fable 5.1 generation of models and can’t substitute for evaluating them; what it tested was the common assumption that “adding a persona makes the model stronger.”

I tried the example from the CS146S slides myself with Fable 5.1: same problem (longest palindromic substring), two personas.

Gen Z digital bestie – conversation log

A helpful assistant that loves programming at the level of a senior software developer – conversation log

This time, both got the program right, and both mentioned the more efficient Manacher’s algorithm unprompted. The most obvious difference was in tone.

This time it looked a lot like cosplay – or VTubing, if you will: the avatar changed, but the person behind it didn’t.

Overusing role prompting is just laziness in disguise. You haven’t thought through the specific requirements of your task and are hoping a generic persona will do the thinking for you. It’s like opening an essay on China’s college entrance exam, the gaokao, with a famous quote: plenty of flourish, little substance specific to the task.

As for why a single persona line changes model behavior at all, I’ll write a separate piece on that – “Why Personas Change Behavior” – when I dive deeper into LLM internals, unpacking conditional probability, attention, and role representations.

When Tricks Become Built-In

A 2022 paper found that adding the prompt “Let’s think step by step” could boost a model’s accuracy on MultiArith from 17.7% to 78.7%. If one sentence is that effective and that broadly applicable, it should end up in the system prompt. Sure enough, models that reason internally later appeared, and OpenAI’s guidance was that there was no longer any need to add this line. What once required a manual nudge had become the model’s default behavior.

Zero-shot and few-shot illustrate this distinction more clearly. Asking a model to translate “今天天气很好” into English without first providing translation examples is zero-shot prompting; giving it a few Chinese-English pairs first and then asking it to translate a new sentence is few-shot prompting. The “shots” here refer to task demonstrations within the prompt, not data the model saw during training. GPT-3’s paper was literally titled Language Models are Few-Shot Learners.

As models get better at understanding instructions directly, you can try some tasks without examples first and see how well the model does; few-shot is far from obsolete. A specific tone of voice, a triple-nested JSON structure, whether an edge case should return an empty array or throw an error – in cases like these, one example can be worth a thousand words of explanation. Show, don’t tell – an old piece of writing advice.

These techniques still help. They are so broadly useful that, as technology advances, they get built into the model itself, the system prompt, or the vendor’s harness. What becomes unnecessary is having to remind the model each time.

The Model Makers’ Job and Our Job

Naval put it more bluntly in his February 2026 podcast A Motorcycle for the Mind: he’s not keen on learning the tips and tricks of how to prompt and which harness to use, because he believes the technology will adapt to him faster than he can adapt to it; in his view, those workflows and tools have a lifespan measured “in weeks, maybe months, not in years.” He did leave one exception: if you’re building a frontier product right now and need every competitive edge, then it’s worth learning. By May, he gave the coding version: skip the Ralph Wiggum, OpenClaw, plan mode advice, throw Codex, Claude, and Gemini at the same problem repeatedly – “waste tokens, save time.”

Following Naval’s line of thinking, I’d split the effort of using AI into two categories: their job and our job. Their job is what model and tool developers are actively improving – making models reason better, use tools more reliably, adapt to how people naturally speak. If a technique merely patches a model’s shortcoming, its value may decline as models improve. Of course, if it helps you get work done today, learning it today is worthwhile. Our job is a different kind of effort. Naval says he doesn’t learn those tricks, but the very next sentence is:

I just can speak English. And for someone like me who is relatively articulate with English and also has a structured mind, and I know how computer architectures work, and I know how computer programs work, and I know how programmers think, then I can actually very precisely specify what I want just through structured English.

That itself is the most powerful prompt technique: clear natural language, a structured mind, knowing how machines work, knowing how programmers think, and then stating precisely what you want. It’s not that Naval has no skills; he developed them long before AI arrived. His prior investment in articulation, programming, and judgment hasn’t just held its value in the AI era – it’s become worth more.

The Prompting Skill That’s Truly Yours: Writing

Tricks that compensate for model weaknesses become unnecessary as models improve, but the ability to clearly express what a task requires never goes out of date.

Think of the model as a temp worker sent by an agency. This person is extraordinarily smart, widely read, and fluent in many languages. But they arrived at your office this morning – they don’t know what your product does, who it’s for, what unwritten rules the team follows, or what caused the last incident. For things they’re already good at, you don’t need to start by teaching them for-loops; what’s worth spending time on is giving them the context specific to this project.

A prompt doesn’t have to be comprehensive from the start. Good prompting isn’t just LLM-friendly – it should be human-friendly too. If every task requires writing a full spec before you begin, you may be put off before you even get started. Evaluating a prompting method should factor in the human cost of getting started.

Mini Habits argues for shrinking the daily action you demand of yourself to something tiny; the Fogg Behavior Model also emphasizes that making a behavior easier makes it more likely to happen. Borrowing this idea, the first step of working with AI can be small too: just say what you want to do. Haven’t thought it through? Let the AI help you work it out by asking questions and trying things with you.

Once you have a first draft, review the results and add requirements. Writing has revision; working with AI is the same. What we should care about includes whether people can start easily and follow through step by step – not just how well the first prompt makes the model answer.

If I were to seriously study prompting, I’d start with the vendor’s official guide: state the task clearly, provide context, give an example when needed. These are the fundamentals worth practicing over time. The official guidance changes too – check the latest version when you need it and judge by results. The generality of AI also means you can use AI to learn AI. Don’t know how to ask a question? Let it help you frame one. Not sure whether something is worth investing time and effort to learn? Ask the AI. You don’t have to finish learning “how to use AI” before you start using AI.

Let them do their job, and we do ours.

Natural Language Is the New Programming Language

Guoliang Chen — Fri, 04 Sep 2026 00:40:00 +0900

Programming Has Always Been a Kind of Writing

I remember the fall of 2022, when I was teaching a friend to write Python. She was the freewheeling type — she’d write whatever felt roughly right, without following Python’s syntax rules. It drove me up the wall. I told her: the compiler can’t understand your free-form expression. It parses code by breaking it down layer by layer, and at the bottom, every piece has to be a valid expression. It might look close enough to a human eye, but to a machine it doesn’t — one missing symbol and the entire syntax tree changes shape.

What happened next, everyone knows. That winter, ChatGPT came out. And just like that, machines really could understand what we say.

The lesson: don’t be too sure of yourself. You think what you said is ironclad truth, but truths change. Because I understood how a Python interpreter works under the hood, I assumed programming in natural language was unrealistic — and missed the fact that the demand for it was very real. It was the same story when ChatGPT first went public: the most dismissive voices among my friends were the NLP researchers, who found it all brute force, no elegance to speak of. The very people who already knew the Transformer architecture were the ones whose expertise blinded them to the magic.

Today, no one doubts that natural language is the new programming language. And the unity of writing and programming was articulated long before — in SICP’s famous line that programs must be written for people to read, and only incidentally for machines to execute. Paul Graham put it more bluntly in Hackers & Painters: hackers and painters are both makers, and the standard for good code is the same as for good prose — clarity, concision, rhythm. I wrote about this very idea in my college Chinese literature final.

The Evolution of Programming Paradigms

In 2017, Karpathy wrote a post called Software 2.0, dividing software into two generations. In his 2025 talk at YC, he added a third. The distinction among the three isn’t what language you write in — it’s how the program comes into being.

1.0: Hand-Written

You figure out the rules first, then type them in line by line as if-else statements. Compilers, operating systems, databases, Excel, TCP/IP — most of the software you use today was made this way. When there’s a bug, you can trace it line by line back to the mistake.

The compiler is 1.0 at its most extreme: a piece of text must parse into a syntax tree with absolute precision — one extra space and it breaks. The upside is determinism: the same input always yields the same output. The cost is that only specialists can write it, and in those days there were never enough software engineers anywhere in the world.

What I told my friend that fall of 2022 was pure 1.0 thinking: machines only understand rules, and humans have to bend to the machine.

2.0: Machine Learning

Some rules, humans simply cannot write. What exactly distinguishes a cat from a dog? What does spam look like? If you can’t write the rules, don’t. Take a different approach: give the system an objective function, a dataset, and a network skeleton, then let gradient descent search for the program in weight space. What it finds is a pile of floating-point numbers that no one can explain. A black box.

Why does it have to be a neural network? Because weight space must be continuous and differentiable for the gradient to move through it — decision trees don’t have that property. Karpathy’s original post put it directly: Software 2.0 is written in the weights of a neural network. Face recognition, speech recognition, machine translation, the recommendation engines behind TikTok and YouTube, AlphaGo — all 2.0. Google Search’s inverted index is 1.0; the RankBrain plugged into its ranking after 2015 is 2.0.

The insight that “rules need not be written by humans” had actually won once before. In The Beauty of Mathematics, Wu Jun recounts how, from the 1950s onward, the mainstream approach to NLP was the rule-based school, following Chomsky’s lead and having linguists hand-write grammars one by one. In the 1970s, IBM’s Frederick Jelinek used statistical methods for speech recognition and drove the error rate down dramatically, leaving behind a memorable line: “Every time I fire a linguist, the performance of the speech recognizer goes up.” The two camps argued for over a decade; by the 1990s, the statistical school had won outright. That was already a victory for not writing rules by hand — only the search method at the time used probabilistic models rather than neural networks.

2.0 has another defining trait: one model does one thing. A face-recognition network can’t play Go; a Go-playing network can’t translate. Switch the problem and you have to collect new data and retrain from scratch. This is ANI — artificial narrow intelligence. You need not just software engineers but also machine learning engineers, plus the data infrastructure to support them. Training a model from scratch as an individual is already impractical.

3.0: Natural Language

Among the programs that 2.0 found, one turned out to be very special: it was trained on essentially all the text on the internet, with syntax, libraries, and every argument on Stack Overflow compressed into its trillions of parameters, and its input is naturally language. You don’t need to retrain it. You just talk to it — your words enter the context window, and its behavior changes. Not a single weight moves.

This is the fundamental break from 2.0: train once, and from then on, switching problems doesn’t require retraining — just a different prompt. ChatGPT, Claude Code — they are this generation. Google Search is shifting too; AI Overviews now sit above the inverted index. The same model writes SQL one moment, polishes a résumé the next, and explains quantum entanglement the moment after that. Whether this counts as AGI is debatable; that “train once, use everywhere” is not.

All you need is someone who can write a prompt. And writing a prompt, at the end of the day, is just saying what you mean clearly. That someone can be you.

Why was it language, of all things, that pulled this off? Because language has always had the power to act on the world. Spells, decrees, contracts — things that, once spoken, change reality. Austin’s 1955 lectures at Harvard were literally called How to Do Things with Words. Code is merely the most rigid variety — so rigid that only specialists could recite the incantations. Naval put it this way: programmers used to be like wizards, needing to memorize arcane spells to cast them; AI amounts to handing everyone a wand. Every problem can be described in language, so a program that has learned language holds the key to every problem.

Karpathy says a prompt is a program. I don’t think that’s a metaphor: a piece of text that changes the behavior of a machine — that is the definition of a program.

The diagram at the top doesn’t quite tell the right story: Software 2.0 and 3.0 didn’t merely eat into the territory of 1.0. What they chiefly did was bring far more new software into existence — especially 3.0, which will create vastly more. Many needs that were previously too costly to serve with software can now be served.

Comparison

	Software 1.0	Software 2.0	Software 3.0
How the program comes into being	Written by humans: figure out the rules, type them as code	Trained: searched from data	Spoken: specified in natural language within the context window
What the program looks like	Source code — deterministic, readable, auditable	Weight matrices — billions of floating-point numbers, uninterpretable	Natural language — tokens in the context window, readable by both humans and machines
What the human does	Requirements → algorithm design → coding → testing	Define a loss function, build a dataset, architect a network, tune hyperparameters	Put intent, constraints, and examples into the context window
Problems one program solves	One	One (ANI)	As many as you like (AGI, in a sense)
Representative products	Compilers, databases, inverted indexes, Excel	Face recognition, speech recognition, recommendation systems, AlphaGo	AI chatbots, coding agents, AI music generators, AI image generators, working agents
People required	Software engineers	Software engineers + ML engineers	Anyone who can say what they mean (product managers)
How to debug	Breakpoints, step-through, read the stack trace	Check data distributions, tune hyperparameters, change the architecture, retrain	Edit the prompt: add context, give examples, rephrase
What failure looks like	Bugs: reproducible, traceable	Generalization failure, overfitting: alchemy that takes experience	Hallucination, misunderstood intent: confidently wrong

The barrier hasn’t disappeared — it’s just moved. 1.0 tested you on algorithms and syntax. 2.0 tested you on data and math. 3.0 tests whether you can make your intent clear.

So how do you train intent and expression? Among so-called “prompt techniques,” which ones expire and which ones don’t? See the next essay.