Prompting, Once and for All: The Skill That Never Fades

Guoliang Chen — Sat, 12 Sep 2026 21:18:00 +0900

Learn Slowly Enough, and You Won’t Have to Learn at All

The previous essay covered the three eras of programming: handwritten code, machine learning, and natural language. Now that natural language has become the new programming language and everyone is a spellcaster, you’re no doubt itching to practice your incantations. But there’s a joke going around in the AI age: “Learn slowly enough, and you’ll never have to learn at all.” And a similar one-liner: “Wake up after a night’s sleep and you’ve missed so much; a year goes by and you haven’t missed a thing.”

Lazy Learning: From JIC to JIT

These jokes are half right. Details that change quickly and are easy to look up when needed can wait.

In programming, there’s a concept called lazy evaluation: don’t compute an expression until someone actually needs its value. Haskell uses this to represent an infinitely long list, evaluating only the parts you actually use. Machine learning has a related concept called lazy learning: a basic k-nearest-neighbors algorithm stores the samples without fitting a parametric model in advance, waiting until a query comes in to compare against them. In the age of AI, there’s more worth learning than ever, and we learn faster than ever – so we should adopt the same philosophy, especially when:

We may not need something soon, and it will be easy to look up and learn when we do.
The current task only requires an understanding of inputs and outputs, so we can treat the rest as a black box for now.
Things change so quickly that details memorized now may need to be relearned later.

As technology advances, the cost of looking things up and learning them changes. A 2011 study in Science found that when people expected to be able to access information later, they remembered the information itself less well but remembered where to find it better. We offload part of our memory to external systems. AI takes this a step further: beyond helping you find information, it can explain things in response to your question, give you examples, and walk you through experiments.

Many people have a hoarding instinct for this kind of knowledge. I suspect there’s a bit of inertia from years of exam culture: too many people only care about whether something would help on a closed-book test, when reality is an open-book exam. We used to learn “just in case” – what if I need this someday? Now we should learn “just in time” – at the moment you need it. Like some JIT compilers: whichever code path runs hot gets the optimization. Wait until you discover that learning something would actually boost your efficiency, and then learn it.

Principles Hold Their Value Longer

Some knowledge is worth learning early, because it keeps helping you evaluate new problems. You don’t need to be able to derive the Transformer, but there are a few mental models that, once built, stay useful.

The model doesn’t see the same characters you do. It first splits text into chunks called tokens – sometimes a word, sometimes half a word. “strawberry” looks like ten letters to you¹, but to the model it might be two or three tokens. That’s why earlier models couldn’t count the r’s reliably: their granularity is different from yours. But this isn’t a ceiling – the tokenization itself doesn’t throw away information, and newer models can perfectly well learn to handle such tasks better. Don’t mistake one generation’s shortcoming for a permanent limit.

Fluency is not correctness. A language model’s basic training objective is “given the preceding text, predict the next most plausible token.” This objective lets it learn vast amounts of knowledge and reasoning patterns, but the training process doesn’t come with a built-in fact-checker. So the smoother it sounds, the more you should stop and ask: can I verify this claim?

“But I already told you” – except the model may no longer see it. Every model has a context window – an upper limit on how much it can take in at once. As a conversation goes on, the earliest exchanges may have slid out of view. You think the conversation history is still there, but from the model’s perspective those messages are no longer in sight. On top of that, the model’s weights don’t update during a conversation – it can pick up a pattern from examples you give in this turn, but close the chat and it forgets, unless the product has a separate memory feature that stores things for you. The model’s knowledge cutoff constrains what it learned during training; it doesn’t stop it from reading material you just pasted in.

Attention Is All You Need came out in 2017, and models have changed enormously since then, but a few concepts – tokens, the context window, the difference between training and inference – still have explanatory power. To judge whether a piece of knowledge is worth learning for the long term, I ask whether it can explain multiple phenomena and whether it continues to help me make judgments after I’ve switched tools. Principles get revised too; a good mental model should also tell you where your own judgments are unreliable.

System Prompts and User Prompts

Let’s start with chatbots like ChatGPT and Claude. When you type “help me write an outline” or “make this shorter” in the text box, that’s a user prompt. The product also provides a system prompt behind the scenes, laying out the assistant’s basic identity, behavioral rules, and so on. To use the temp worker analogy: the system prompt is the company handbook; the user prompt is the specific task you hand them. For a fuller guide to writing prompts, check out the community-maintained collection of leaked system prompts. As an aside, I believe users should have a right to know about these trust-shaping instructions, just as mainstream general-purpose compilers are almost all fully open source.

For day-to-day chatting, just state your need in the text box. Within a single conversation, earlier messages typically remain as context for subsequent responses – you don’t have to start from scratch every turn. If you have preferences you keep repeating, like “respond in Chinese, keep technical terms in English,” you can save them in ChatGPT’s custom instructions. Background and requirements that apply only to a particular set of conversations can go into Claude’s project knowledge and project instructions.

These are personalization features offered by the product – not the same as editing the vendor’s system prompt directly. For users, it’s enough to start by stating the task at hand clearly; whatever you find yourself repeating, save it and stop repeating it. The system prompt always takes priority over the user prompt.

How prompts are arranged inside agents is considerably more complex – I’ll cover that in a future piece.

Role Prompting Is Just Cosplay Now

With that division of responsibilities in mind, it is easier to judge what work popular prompting techniques still require us to do.

A slide from Stanford’s fall 2025 CS146S: The Modern Software Developer reads Use role prompting aggressively to make system prompts more powerful. Role prompting means opening with something like “You are a top-tier frontend expert.” Many people treat it as a mandatory opener, as if the model wouldn’t know how to write frontend code without it.

With this generation of thoroughly post-trained assistant models – Fable 5.1 / GPT-6 Astra – my default is to state the task directly and skip the preamble. Understanding the problem carefully, applying relevant knowledge, and giving a helpful answer are precisely the behaviors the labs are training into these models. Much of the work of shaping a model into a good assistant has already been done before it reaches your hands.

Through post-training methods like instruction tuning and RLHF, labs fine-tune models with demonstration responses and preference feedback to make them better at following requests and helping users. “Be a helpful assistant” has become the model’s learned default behavior – users don’t need to teach it from scratch in every conversation. Anthropic’s persona selection model offers a useful frame: post-training shapes and refines the Assistant persona.

Even without specifying a persona, you’re already talking to a model deliberately shaped to be an assistant. The task “build me a React web page” itself already provides a frontend development context. Whether a line like “you are a top-tier frontend expert” adds value depends on what it contributes beyond this baseline – not merely on whether the output changes.

This is why I skip the generic persona: what it’s trying to remind the model to do is very likely already built into the default assistant behavior and the task description. “Already tuned” here means behavior optimized toward helping users across a broad range of tasks; whether a specific task turns out well still comes down to results. So I state the task first, and add requirements only when I spot gaps – rather than treating “you are an expert” as a prerequisite.

If I want it to “act like an interviewer who asks one question at a time,” or “evaluate this page from an accessibility auditor’s perspective,” that’s different: I’m specifying the mode of collaboration or the scope of review. That information can’t necessarily be inferred from “help me build a web page.” A persona can still serve as shorthand for requirements like these; what makes it useful is the task information it adds.

Existing research also cautions against putting too much faith in titles. Zheng et al.’s study, first published in 2023 and finalized in 2024, tested 162 personas in system prompts across nine models from four model families on 2,410 MMLU multiple-choice questions – and found no stable improvement in overall accuracy from adding a persona. This study didn’t test the Fable 5.1 generation of models and can’t substitute for evaluating them; what it tested was the common assumption that “adding a persona makes the model stronger.”

I tried the example from the CS146S slides myself with Fable 5.1: same problem (longest palindromic substring), two personas.

Gen Z digital bestie – conversation log

A helpful assistant that loves programming at the level of a senior software developer – conversation log

This time, both got the program right, and both mentioned the more efficient Manacher’s algorithm unprompted. The most obvious difference was in tone.

This time it looked a lot like cosplay – or VTubing, if you will: the avatar changed, but the person behind it didn’t.

Overusing role prompting is just laziness in disguise. You haven’t thought through the specific requirements of your task and are hoping a generic persona will do the thinking for you. It’s like opening an essay on China’s college entrance exam, the gaokao, with a famous quote: plenty of flourish, little substance specific to the task.

As for why a single persona line changes model behavior at all, I’ll write a separate piece on that – “Why Personas Change Behavior” – when I dive deeper into LLM internals, unpacking conditional probability, attention, and role representations.

When Tricks Become Built-In

A 2022 paper found that adding the prompt “Let’s think step by step” could boost a model’s accuracy on MultiArith from 17.7% to 78.7%. If one sentence is that effective and that broadly applicable, it should end up in the system prompt. Sure enough, models that reason internally later appeared, and OpenAI’s guidance was that there was no longer any need to add this line. What once required a manual nudge had become the model’s default behavior.

Zero-shot and few-shot illustrate this distinction more clearly. Asking a model to translate “今天天气很好” into English without first providing translation examples is zero-shot prompting; giving it a few Chinese-English pairs first and then asking it to translate a new sentence is few-shot prompting. The “shots” here refer to task demonstrations within the prompt, not data the model saw during training. GPT-3’s paper was literally titled Language Models are Few-Shot Learners.

As models get better at understanding instructions directly, you can try some tasks without examples first and see how well the model does; few-shot is far from obsolete. A specific tone of voice, a triple-nested JSON structure, whether an edge case should return an empty array or throw an error – in cases like these, one example can be worth a thousand words of explanation. Show, don’t tell – an old piece of writing advice.

These techniques still help. They are so broadly useful that, as technology advances, they get built into the model itself, the system prompt, or the vendor’s harness. What becomes unnecessary is having to remind the model each time.

The Model Makers’ Job and Our Job

Naval put it more bluntly in his February 2026 podcast A Motorcycle for the Mind: he’s not keen on learning the tips and tricks of how to prompt and which harness to use, because he believes the technology will adapt to him faster than he can adapt to it; in his view, those workflows and tools have a lifespan measured “in weeks, maybe months, not in years.” He did leave one exception: if you’re building a frontier product right now and need every competitive edge, then it’s worth learning. By May, he gave the coding version: skip the Ralph Wiggum, OpenClaw, plan mode advice, throw Codex, Claude, and Gemini at the same problem repeatedly – “waste tokens, save time.”

Following Naval’s line of thinking, I’d split the effort of using AI into two categories: their job and our job. Their job is what model and tool developers are actively improving – making models reason better, use tools more reliably, adapt to how people naturally speak. If a technique merely patches a model’s shortcoming, its value may decline as models improve. Of course, if it helps you get work done today, learning it today is worthwhile. Our job is a different kind of effort. Naval says he doesn’t learn those tricks, but the very next sentence is:

I just can speak English. And for someone like me who is relatively articulate with English and also has a structured mind, and I know how computer architectures work, and I know how computer programs work, and I know how programmers think, then I can actually very precisely specify what I want just through structured English.

That itself is the most powerful prompt technique: clear natural language, a structured mind, knowing how machines work, knowing how programmers think, and then stating precisely what you want. It’s not that Naval has no skills; he developed them long before AI arrived. His prior investment in articulation, programming, and judgment hasn’t just held its value in the AI era – it’s become worth more.

The Prompting Skill That’s Truly Yours: Writing

Tricks that compensate for model weaknesses become unnecessary as models improve, but the ability to clearly express what a task requires never goes out of date.

Think of the model as a temp worker sent by an agency. This person is extraordinarily smart, widely read, and fluent in many languages. But they arrived at your office this morning – they don’t know what your product does, who it’s for, what unwritten rules the team follows, or what caused the last incident. For things they’re already good at, you don’t need to start by teaching them for-loops; what’s worth spending time on is giving them the context specific to this project.

A prompt doesn’t have to be comprehensive from the start. Good prompting isn’t just LLM-friendly – it should be human-friendly too. If every task requires writing a full spec before you begin, you may be put off before you even get started. Evaluating a prompting method should factor in the human cost of getting started.

Mini Habits argues for shrinking the daily action you demand of yourself to something tiny; the Fogg Behavior Model also emphasizes that making a behavior easier makes it more likely to happen. Borrowing this idea, the first step of working with AI can be small too: just say what you want to do. Haven’t thought it through? Let the AI help you work it out by asking questions and trying things with you.

Once you have a first draft, review the results and add requirements. Writing has revision; working with AI is the same. What we should care about includes whether people can start easily and follow through step by step – not just how well the first prompt makes the model answer.

If I were to seriously study prompting, I’d start with the vendor’s official guide: state the task clearly, provide context, give an example when needed. These are the fundamentals worth practicing over time. The official guidance changes too – check the latest version when you need it and judge by results. The generality of AI also means you can use AI to learn AI. Don’t know how to ask a question? Let it help you frame one. Not sure whether something is worth investing time and effort to learn? Ask the AI. You don’t have to finish learning “how to use AI” before you start using AI.

Let them do their job, and we do ours.

Prompt on Message from Garden Place