What Belongs in a System Prompt, and What Does Not

Your application answers correctly nine times, then does something odd on the tenth. Same code, same model, a slightly different question. You add a line to the system prompt telling it not to do that. It obeys for a week. Then a user pastes something into a form and the behaviour comes back.

Almost always, the problem is not that the instruction was badly written, the same way a disappointing chat answer usually traces to a missing constraint rather than to poor phrasing. It is that the instruction was in the wrong place, competing with text that outranked it. There is a documented hierarchy governing which text wins, and once you can see it, this class of bug stops being mysterious.

What is a system prompt actually for?

It holds what is true for every request your application will ever make, which is also where the role sentence belongs among the five prompt patterns that fix most prompts. The role the model is playing, the constraints it must never break, the format you always want back, and the context that does not change between users.

Anthropic's prompting guidance is direct that a role set in the system prompt focuses behaviour and tone, and that a single sentence measurably changes output. Their example is exactly one line: "You are a helpful coding assistant specializing in Python." That is not a warm-up. It is the whole technique, and most system prompts would work better if they were closer to that length.

The failure mode is the opposite. System prompts grow because every bug produces a new line, and after six months you have 900 words in which half the rules contradict the other half, and nobody knows which ones the model is still reading. That bloat is also billed on every turn, which is one of the categories we count in a breakdown of where your tokens actually go.

What belongs where?

Four places text can live, four different jobs. Putting something in the wrong one causes a specific, predictable failure rather than general vagueness.

The third row is the one teams get wrong most often, and it is the expensive one. Guidance about when to use a tool belongs in the tool's own description, where the model reads it at the moment it is choosing. Moved into the system prompt, that same sentence sits hundreds of tokens away from the decision it is supposed to inform, which is exactly the positioning problem described in what to cut when the context window fills.

Why does the model ignore your instruction?

Because something with more authority said otherwise, and the ranking is published. OpenAI's Model Spec dated 18 December 2025 sets out five levels that resolve conflicts between instructions.

Two rules inside that structure do most of the practical work. Conflicts at the same level are resolved by recency, so a later message beats an earlier one. And assistant and tool messages carry no authority whatsoever, with untrusted content such as quoted text and multimodal input treated as informational unless you explicitly delegate authority to it.

Read that second rule again if you have ever wondered why a model followed an instruction hidden in a document it retrieved. Under the spec, it should not have. Text arriving through a tool result is data about the world, not a command, and a system that treats it as a command has a security problem rather than a prompting problem. We went through the practical shape of that attack in our piece on what six thousand prompt injection attempts revealed.

That analogy is the cleanest test available. If the text would be a parameter, it is a user turn. If it would be part of the function body, it is a system prompt. Anything you find yourself string-formatting into the system prompt on every request is probably an argument in disguise.

Three real system prompts, and what they are doing

Public examples are scarce, which is why so much advice in this area is invented. These three are documented by their vendors.

The consumer assistant prompt

Anthropic publishes the system prompts used by claude.ai and the mobile apps, with dated entries per model, the most recent being Claude Opus 5 on 24 July 2026. The page carries an important caveat: these apply to the consumer products only, never to the API. If you build on the API you inherit none of it, which surprises people who assume the model arrives pre-configured.

The published structure is worth copying even if the content is not yours to use. It covers the current date, product information, refusal handling, child safety, tone and formatting preferences, user wellbeing, and the knowledge cutoff. Every one of those is a rule true for every conversation, which is the definition from the section above, applied consistently. Notice what is absent: nothing about any particular user, and no task-specific instruction.

The one-line role prompt

The API example is a single sentence naming a role and a specialisation. It is the smallest system prompt that does real work, and it is a better starting point than a template. Write one sentence, ship it, and add a line only when a specific failure demands it. A prompt that grew from evidence is testable. A prompt that started at 500 words never is.

The long-horizon agent prompt

The third is the most interesting because it addresses a failure that only exists in loops. Anthropic's guidance includes a sample instruction telling the model that its context window will be compacted automatically, that it should not stop early over token budget concerns, and that it should save progress to memory before the window refreshes.

That paragraph exists because recent models track their own remaining context. Left uninformed, a model approaching its limit starts wrapping up: summarising instead of finishing, declaring partial success, tidying rather than working. The instruction is not motivational. It is telling the model a true fact about its environment that it cannot otherwise observe, which is the highest value thing a system prompt can do.

Does forceful language help?

It used to, and the documentation now says it backfires. This is the single most useful recent change in prompting practice and almost nobody has updated for it.

Anthropic's guidance states that recent Claude models are more responsive to the system prompt than earlier ones, and that prompts written to stop tools being underused may now cause overtriggering. The recommended fix is to dial the language back: where you once wrote that the model absolutely must use a tool in capital letters, plain phrasing such as "Use this tool when..." now works better.

The mechanism is intuitive once stated. Emphatic language was compensation for a model that under-weighted instructions. Remove the under-weighting and the compensation becomes distortion. If your prompt is full of capitalised imperatives written against a model from two years ago, they are now actively steering behaviour away from what you want, and deleting them is a change worth testing before you write anything new.

What changes when the model runs in a loop?

The system prompt stops describing a task and starts describing a working environment. Four additions earn their place, and they are all environmental facts rather than instructions about quality.

• What happens to context. Whether it compacts, whether state can be written to disk, and whether stopping early is ever the right move.
• How to persist state. Anthropic's guidance recommends structured formats such as JSON for things like test status, freeform text for progress notes, and git as a checkpoint log the model can read on a fresh window.
• How to verify. As autonomous runs lengthen, the model needs a way to check correctness without a human, so the prompt should name the verification tools it has.
• How to start a fresh window. Being prescriptive here works better than being general: check the working directory, read the progress file, run one integration test before writing anything new.

There is a counterintuitive recommendation in the same guidance worth flagging. When a context window is cleared, starting completely fresh can beat compaction, because recent models are effective at rediscovering state from the filesystem. A clean start with good instructions can carry less confusion than a summary of a confused session.

If those tools reach outside the codebase, the tool descriptions matter as much as the prompt. Our walkthrough of setting up an MCP server covers writing those descriptions, and our overview of what MCP makes possible covers why the standard exists.

How do you steer how eagerly the model acts?

With one paragraph, in the system prompt, and it is the highest leverage block most applications never write.

Anthropic's guidance publishes both directions as sample prompts. To make the model proactive, you tell it to implement changes rather than only suggest them, and to infer the most useful likely action when intent is unclear, using tools to discover missing details instead of guessing. To make it conservative, you tell it not to change files unless clearly instructed, and to default to research and recommendations when intent is ambiguous.

Those two paragraphs produce visibly different products from the same model. A support assistant that acts on an ambiguous request is a liability. A coding agent that only ever suggests is an expensive autocomplete. Most teams never write either paragraph, inherit whatever default the model has, and then attribute the resulting behaviour to the model's personality.

Write the one that matches your risk profile, put it near the top, and revisit it whenever you change models. The default eagerness is not a constant across model versions, which is exactly the lesson of the overtriggering guidance above.

What about the things the model cannot know?

These are the highest value lines in any system prompt, and they are pure fact rather than instruction.

The date is the classic one. A model's knowledge stops at its cutoff, and the published consumer prompts include the current date for exactly this reason. Without it a model reasons about "now" using a date months in the past, which quietly corrupts anything involving recency, deadlines or arithmetic on time.

The environment is the second. Which tools exist, what they cost, whether a command is reversible, which directory is writable, whether the work is being reviewed by a human before it ships. None of that is inferable from the conversation, and every one of those facts changes what a reasonable action looks like.

The product is the third. What your application is, who uses it, what it does not do. A model asked something outside scope will answer helpfully and wrongly unless the boundary is stated. One sentence naming what the product is not prevents more bad output than three paragraphs about tone.

A useful way to audit this: read your system prompt and mark every sentence as either a fact the model could not otherwise know, or an instruction about how to behave. Facts almost always earn their tokens. Instructions are where the bloat is, and where contradictions accumulate.

How do you know a change actually helped?

You measure it, because your intuition about prompt edits is unreliable in a specific, documented way.

The ProSA framework, published in October 2024 by Jingming Zhuo and colleagues, introduces a metric called PromptSensiScore to quantify how much a model's performance moves when a prompt is rephrased without changing its meaning. Their findings are the practical part: sensitivity varies by dataset and by model, larger models are more robust than smaller ones, few-shot examples reduce the effect, and higher model confidence correlates with greater robustness. Subjective evaluation is itself sensitive to prompt wording, particularly on reasoning-heavy tasks.

Three consequences follow directly. First, a single before-and-after test tells you almost nothing, because the variance between two phrasings of the same instruction can exceed the effect you are trying to measure. Second, if you are chasing consistency, adding two or three examples to the prompt is better supported by evidence than adding another rule. Third, results from a small model do not transfer upward, so a prompt tuned on a cheap model may be carrying compensations the larger one does not need.

The practical minimum is a fixed set of twenty inputs with known-good outputs, run before and after every prompt change. Twenty is small enough to maintain and large enough that a real regression shows up. If that sounds like too much work for a prompt edit, that is the honest cost of the claim that the edit improved anything.

A checklist for the prompt you have now

Open your current system prompt and go through this once. It takes about twenty minutes and usually removes a third of it.

Delete anything that is true for only some requests. Move anything about when to use a tool into that tool's description. Remove capitalised imperatives and see whether behaviour actually degrades, because on recent models it often improves. Check that nothing arriving from a tool or a retrieved document is being concatenated into the system prompt, because that is the injection path. Tell the model what it cannot observe: the date, the environment, whether context compacts, what tools verify its work.

Then stop adding. The most common shape of a broken system prompt is not a missing rule. It is nine months of accumulated rules, each one added to fix a specific incident, none ever removed, several now contradicting each other in ways the model resolves by picking one at random.

The short version

A system prompt is the constant part of your application, expressed in words. Rules that never change go in it. Requests go in the user turn. Tool guidance goes in tool descriptions. Anything the model retrieved is data with no authority, and treating it otherwise is how injections work.

Write it short, keep it in version control, drop the shouting, and measure changes against a fixed set of examples rather than a hunch. If you are pricing what all this costs to run once it works, our plans and credits page shows the per project view rather than the per token one.