- Anthropic's own prompting reference puts the useful number at three to five examples, and asks that they be relevant and diverse rather than numerous.
- A peer reviewed study found that randomly replacing the labels in your examples barely hurt performance across twelve models, which reframes what an example is for.
- What demonstrations actually supply, per that study, is the label space, the distribution of the input text and the format of the sequence.
- That explains the ceiling. Once the format and the range are established, a sixth example adds nothing new and still bills on every request.
- The right question is never how many examples. It is how many distinct kinds of input you have, because each example should buy one.
- Examples cannot fix missing knowledge, an ambiguous task definition or a model that is too small for the job. Those look like the same failure and are not.
- Fine tuning moves the cost from every request to one training run, which only pays back above a volume most teams never reach.
Your prompt handles the first three cases you tested and falls over on the fourth. The obvious next move is to add the fourth case as an example, and it usually works, which is also why examples from your own catalogue beat anything a prompt generator can invent for you. The question nobody answers is what happens when you keep going, because the advice in circulation is a shrug: add a few, more is better, see how it goes.
There is a better answer available, and it comes from two places. One is vendor guidance that names an actual number. The other is a study that explains why that number exists rather than being folklore.
Where did few shot prompting come from?
From the paper that named the technique, and the origin matters because it explains the framing everyone inherited.
The 2020 paper Language Models are Few-Shot Learners, by Tom B. Brown and a large team, introduced a 175 billion parameter model and made a specific argument: scaling a language model greatly improves task agnostic few shot performance, sometimes reaching competitiveness with fine tuned approaches. The framing in the abstract is the part that stuck. Humans can perform a new language task from only a few examples or from simple instructions, and the paper positions few shot prompting as the machine version of that. Crucially, the model was applied without any gradient updates or fine tuning, with the demonstrations specified purely as text.
That is where the word learning in in context learning comes from, and it is slightly misleading. Nothing is learned in the sense of being retained. The examples sit in the prompt, are processed on that request, and are gone. Every subsequent request pays for them again. Understanding that one fact removes most of the confusion about when examples are worth it.
How many examples should you actually use?
Three to five, according to the vendor that publishes a number, and the reason the number is small is more interesting than the number itself.
Anthropic's prompting best practices reference states it directly: include three to five examples for best results. It attaches three conditions rather than treating count as the whole story. Examples must be relevant, mirroring your actual use case closely. They must be diverse, covering edge cases and varying enough that the model does not pick up an unintended pattern. And they must be structured, wrapped in example tags so the model can tell them apart from your instructions.
OpenAI's prompt engineering guide does not name a count but points at the same property: show a diverse range of possible inputs with the desired outputs. Both are saying that the axis is coverage, not quantity. Five examples of the same kind of input are one example repeated five times, and cost five times as much.
Why do examples stop helping?
Because of what they are teaching, which turns out not to be the answers. This is the finding that reframes the whole technique.
The study Rethinking the Role of Demonstrations, by Sewon Min, Xinxi Lyu, Ari Holtzman and colleagues, submitted in February 2022 and revised that October, ran an experiment that sounds like it should break everything. They randomly replaced the labels in the demonstrations, so the examples showed wrong answers. Performance barely moved, across a range of classification and multiple choice tasks, consistently over twelve different models.
Their conclusion names what the demonstrations are supplying instead: examples of the label space, the distribution of the input text, and the overall format of the sequence. Not the mapping between input and answer, which is what everyone assumes they are demonstrating.
Follow that through and the ceiling explains itself. The label space is established as soon as the model has seen each possible answer once. The format is established after one example. The input distribution needs a handful, because that is the only one of the three that requires variety. Once all three are covered, another example adds a fourth copy of information already conveyed, and the price is charged on every call for as long as the feature exists.
The label result is easy to over read. It does not mean your examples can be wrong. It means their correctness is not where the lift comes from, so effort spent perfecting answers is misallocated against effort spent covering unusual inputs. Wrong labels in a prompt a human will also read are still a bad idea for ordinary reasons.
Zero, one, few and fine tuning compared
Four levels, and they differ on more than count. The last column is the one that decides most real cases.
| Approach | What it establishes | What it cannot fix | What it costs |
|---|---|---|---|
| Zero shot | Nothing beyond your written instruction. The model infers format from wording | Format drift, house conventions, anything you did not describe precisely | Nothing extra. The cheapest possible call |
| One shot | Format and the shape of a valid answer | Edge cases, unusual inputs, the range of things you actually receive | One example on every request, forever |
| Few shot, three to five | Format, label space and a usable slice of the input distribution | Missing knowledge, an ambiguous task, a model too small for the job | Three to five examples on every request. The largest recurring prompt cost most features carry |
| Fine tuning | All of the above, folded into the weights rather than the prompt | The same three. It changes where the cost sits, not what examples can do | A training run, a dataset, and a model you now have to version and re run when the base changes |
The fine tuning row deserves a sentence of caution because it is the one most often reached for too early. It moves a per request cost to a one off cost, which is arithmetic that only pays back above a request volume most small teams never hit. It also creates an artefact you own and maintain. The comparison against retrieval, which is the other way people add knowledge, is worked through in our piece on choosing between retrieval and fine tuning.
A worked progression on one task
The task: read a customer message and label it as refund, delivery, product question or other. Below is what the prompt contains at each level, and what is genuinely established by that point. These are not measured accuracy figures, because we did not run the experiment, and inventing a curve would be worse than useless.
At zero examples, the prompt is the instruction and the four label names. The model will produce labels. It will also occasionally produce "Refund request" instead of "refund", or add a sentence explaining its choice, because nothing told it not to. Everything downstream that expects one of four exact strings now needs a normalisation step.
At one example, that class of problem disappears. A single input and output pair fixes the casing, the exact token and the absence of explanation. This is the largest single improvement in the whole progression, and it is delivered by the cheapest addition.
At three examples, you can cover distinct kinds of input rather than three variations of one. A short blunt message, a long rambling one, and one that mentions two topics but is really about a third. The third is the one earning its place, because ambiguity is where a classifier fails and where your instruction was silent.
At eight examples, you are almost certainly repeating. Unless your inputs genuinely fall into eight distinguishable shapes, examples six through eight are teaching the format for the eighth time. They are also sitting in the input on every call, which on a queue processing thousands of messages a day is a real number rather than a rounding error, as the piece on why an LLM bill comes in over the estimate works through.
The rule that falls out is simple enough to apply without thinking about it. Add an example only when it covers a kind of input none of the existing ones cover. If you cannot say which new kind it covers, it is a copy.
When are examples the wrong tool entirely?
In three situations that look identical from the outside, which is why people keep adding examples to problems examples cannot touch.
The model lacks the knowledge. If your task requires facts about your products, your policies or your customers, no number of examples supplies them, because examples demonstrate form rather than content. The fix is putting the information in the prompt or retrieving it. Adding a twelfth example here is the most common wasted afternoon in this whole area.
The task is genuinely ambiguous. If two reasonable people would label the same input differently, examples will not resolve it, they will encode whichever answer you happened to pick and then apply it inconsistently. The fix is upstream: define the boundary, in words, in the instruction.
The model is too small. Below a certain capability, a task fails regardless of demonstration. The tell is that the answers are wrong in varied and creative ways rather than consistently wrong in one way. Consistent wrongness responds to examples. Varied wrongness usually does not.
Distinguishing these takes one experiment. Put the answer in the prompt for a case that is failing and see whether the model gets it right. If it does, you have a knowledge problem, not a demonstration problem. If it still does not, you have a capability or definition problem, and more examples will burn tokens without moving anything.
Do examples interact with reasoning models?
Yes, and the interaction runs in a direction that catches people out when they change model, the same reversal that decides whether chain of thought prompting is worth its tokens.
OpenAI's guide draws the distinction plainly: its standard models benefit from precise instructions that spell out the required logic and data, while its reasoning models work better with high level guidance rather than explicit step by step instruction. Examples that walk a model through a procedure are a form of explicit step by step instruction. On a model that reasons natively, they can constrain a process the model would have run better unguided.
Anthropic's reference points at the compatible version of this. It notes that multishot examples work with thinking, and recommends putting thinking tags inside your few shot examples to show the reasoning pattern, which the model then generalises to its own thinking. That is a meaningfully different instruction from demonstrating an answer: you are demonstrating an approach.
The practical consequence is that an example set tuned on one model is not portable. When you swap models, the examples need re checking along with everything else, which is one of several reasons a fixed test set is worth more than a good prompt. Our note on how to tell whether a prompt change actually helped covers building one cheaply.
Does the order of the examples matter?
It can, and the mechanism is worth understanding even though the fix is trivial.
If the demonstrations supply the distribution of the input text and the label space, then a set of examples that happens to end on the same label three times has told the model something about frequency that you did not intend. The same applies to length: five short examples followed by a long real input is a mismatch between what was demonstrated and what is being asked. Anthropic's guidance addresses this indirectly by asking for diversity specifically so the model does not pick up an unintended pattern, which is exactly the failure this describes.
The cheap defence is to arrange examples so that no obvious sequence exists. Alternate labels rather than grouping them. Vary length deliberately. And put the hardest case somewhere other than last, because a final example carries more weight in a prompt than a middle one for the same reason a final instruction does.
None of this is worth agonising over on a three example prompt, where there is barely a pattern to pick up. It becomes worth twenty seconds of attention as the set grows, which is one more reason the set should not grow much.
What does this mean if you are not a developer?
The same thing, minus the token arithmetic. If you use an assistant in a browser rather than through an API, examples still work exactly as described and cost you nothing you can see.
The practical version for someone running a business is a saved block of three examples of your own past work, pasted at the top of any recurring task. Three real product descriptions you wrote and liked. Three replies you sent to awkward customer emails. Three social posts that performed. That block does more for output quality than any amount of instruction about tone, because it demonstrates the register instead of describing it, and because it carries specifics no description could.
The same ceiling applies. A fourth description that resembles the first three teaches nothing. A fourth that covers a completely different product category teaches something. And the same limits apply too: if the assistant does not know your delivery policy, showing it three well written replies will not tell it what the policy is.
Writing examples that earn their tokens
Four habits, all cheap, all derived from the sources above rather than from taste.
- Pick inputs by difficulty, not by tidiness. The instinct is to use clean examples. The study says the input distribution is what transfers, so a messy real message teaches more than an idealised one.
- Fence them. Wrap examples in tags so they are not mistaken for instructions or for the input you want processed. This is the same fencing discipline that matters throughout a prompt, covered among the five prompt patterns worth knowing.
- Vary the answers across the label space. If four labels are possible and your examples only ever show two, you have demonstrated a smaller label space than the task has, and the model will lean toward what it saw.
- Delete one and check. Removing an example and finding nothing changes is the only reliable way to learn that it was a copy. Nobody does this, which is why example sets only ever grow.
Where this fits in a real application
In a chat window, an extra example costs you nothing you notice. In an application, it is a fixed tax on every request, so the calculus changes and it is worth being deliberate.
The two levers that reduce the tax are worth knowing. Prompt caching means a stable example block can be billed at a fraction of the normal input rate, which changes the arithmetic on a large example set substantially, and the comparison of what a real feature costs across vendors covers how those rates work. And if what you actually need from examples is a guaranteed output shape rather than a demonstrated one, that job belongs to schema enforcement rather than demonstration, as set out in how to force a model to hold a schema. Examples make the shape likely. A schema makes it certain, and it costs less.
If you are building the surrounding product rather than a script, the same discipline applies to every part of the prompt that repeats, and our plans and credits page lays out what that costs when the build runs on our platform instead of your own plumbing.
The answer, in one paragraph
Use as many examples as you have distinct kinds of input, which for most tasks is three to five. Choose them for coverage rather than correctness, because the research says correctness is not what transfers. Stop when a new example cannot be justified by naming the input type it adds. And when examples are not fixing the problem, check whether the problem is knowledge, definition or capability before adding a sixth, because those three account for nearly every case where more examples made no difference at all.