- Most people use a chat assistant as a smarter search box, one question at a time, and throw the thread away. Almost all of the value sits in the things that survive the thread.
- Five capabilities carry the difference: attaching the real file, correcting in place instead of restarting, writing standing instructions, asking for a fixed shape rather than prose, and knowing when the job belongs in an API call.
- Asking for a shape is not a developer trick. The same instinct that makes a schema work in code makes a table work in a chat, because a named field is a harder target to miss than a paragraph.
- The honest limits are documented, not folklore. A 2025 OpenAI paper argues models guess rather than admit uncertainty because evaluations reward confident answers.
- Arithmetic is the sharpest example. One study found a model at 99.81 percent on ordinary two integer addition collapsing to 7.51 percent when the same sums used unfamiliar symbols, which is pattern matching rather than rule following.
- Move a task to the API when you run it more than about once a week with the same instructions. Before that, the chat box wins on speed of iteration.
Watch somebody experienced use a chat assistant and the surprising thing is how little of it looks like conversation. They paste a spreadsheet. They say "no, the third one is wrong, redo just that". They keep one thread open for three days. They rarely write anything that resembles a question.
Watch most people and it is one question, one answer, close the tab. That is a search box with better manners, and it is roughly a tenth of what the thing does. This walkthrough covers the five capabilities that separate those two behaviours, each with the before and the after, plus an honest list of what stays broken no matter how well you ask.
Why does giving it the file beat describing the file?
Because a description is your summary of the data, and your summary already contains your assumptions. Attach the actual file and the model reads what is there, including the rows that break your mental model of your own business.
Before. "I sell about forty products, mostly homewares, average order around 45 euros. Which categories should I push in the autumn?" The answer will be competent and generic, because everything in it came from your one sentence summary.
After. Export twelve months of orders as a CSV, attach it, and ask the same question. Now the reply can tell you that one product accounts for a third of returns, that a category you think is small has the best repeat rate, and that a price point you keep raising is where the drop off starts. None of that was in your sentence, because you did not know it.
The habit worth building is to attach before you explain. A file, a screenshot of the analytics page, the actual email a customer sent. The quality difference does not come from the model being cleverer with files. It comes from you no longer being the bottleneck on what it knows.
What does correcting in the thread do that starting over does not?
It keeps everything that was already right. Starting a new chat throws away the working half of the answer along with the broken half, and then you spend three messages rebuilding context you already had.
Before. The product description comes back too long and too excited. You open a new chat and write a longer prompt, hoping to prevent both problems at once. The new draft fixes the length and loses the one good line from the first attempt.
After. You stay in the thread. "Cut it to eighty words. Keep the second sentence exactly. Drop every adjective you would not use about a kettle." Each instruction is small and checkable, and it builds on a draft that already exists. Three rounds of this beats one enormous prompt, and it takes less typing.
Anthropic's prompting guidance puts the underlying rule well: treat the model as a capable colleague with no context on how you work, and remember that clarity beats brevity. A vague long prompt loses to a specific short one. The thread is where specificity is cheapest, because you can point at something concrete and say fix that.
Where do custom instructions actually pay off?
On the corrections you make every single time. If you have told it twice this week that you sell to trade customers and not consumers, that sentence belongs in your standing instructions rather than in your typing fingers.
Before. Every session starts with two paragraphs of context. What the business is, who buys, what tone, which words are banned. You write it slightly differently each time, so you get slightly different behaviour each time and cannot tell whether the model changed or you did.
After. That context lives in the settings and applies to new conversations without being retyped. What goes in it: what you sell, who to, the words you never want to see, the format you usually want, and the correction you find yourself making most often. What does not go in it: anything task specific, because a standing instruction that only applies to a quarter of your work quietly damages the other three quarters.
This is the same discipline as writing a system prompt for an application, at a smaller scale, and the failure modes are identical. We went through them in our guide to what belongs in a system prompt and what does not, and the main one is worth repeating here: instructions accumulate, nobody ever deletes one, and after six months half of them contradict each other.
Why ask for a table when prose reads better?
Because prose hides omissions and a table shows them. If you ask for a paragraph about five suppliers and it covers four, you may not notice. If you ask for five rows with a delivery time column and one cell is blank, you notice immediately.
Before. "Summarise these twenty customer emails." You get four smooth paragraphs. They are accurate and completely unusable, because there is nothing to sort, count or act on.
After. "For each email give me one row: customer, issue category from this list, sentiment, whether they asked for money back, one line quote." Now you have something you can paste into a sheet and sort by category, and the categories are yours rather than invented on the fly.
This has a formal version on the developer side. OpenAI's structured outputs documentation describes constraining generation to a JSON schema so the response cannot omit a required key or invent an enum value, with a separate refusal field so a safety refusal is detectable in code rather than parsed out of prose. The docs are also honest that a schema constrains shape and not truth: the output can still be wrong, just correctly wrong. If you build on this, our piece on forcing a model to hold a shape covers the failure cases in practice.
In the chat box you get most of the benefit for free by naming your columns. The mechanism is the same. A named field is a target the model can miss visibly.
What is it still bad at?
Three things, all documented, none of which improve because you prompt better.
Confident guessing. The 2025 paper Why Language Models Hallucinate makes an uncomfortable argument: models guess when uncertain because the way they are graded rewards it. A benchmark scores a wrong answer and an admission of ignorance identically, at zero, so a system optimised for score learns to always produce something. The authors describe it as being optimised to be a good test taker. Read that as a permanent property of anything trained this way, not a bug awaiting a patch.
Arithmetic. Researchers testing whether models grasp addition or merely recognise it found accuracy of 99.81 percent on ordinary two integer sums collapsing to 7.51 percent when the same arithmetic was written with unfamiliar symbols. Same operation, same difficulty, different surface. They also recorded commutativity violations up to 20 percent, meaning the model produced different results for the same sum in a different order. Do not let it total your invoices.
Anything after its cutoff. Unless it searches, it does not know. This is obvious and still catches people on prices, on policy, on which version of a platform does what.
| Task | Chat box | Why |
|---|---|---|
| First draft of copy you will edit | Strong | Volume and variation are what it is best at |
| Sorting and tagging messy text | Strong | Judgement on language, checkable output, no arithmetic |
| Turning a document into a summary you can act on | Strong | You supply the source, so its knowledge is not load bearing |
| Any calculation you will not verify | Weak | Pattern matching rather than rule following, per the addition study |
| Facts about prices, policies or recent events | Weak | No knowledge past the cutoff unless it explicitly searches |
| Telling you when it does not know | Weak | Training rewards a confident guess over an admission |
The capability nobody mentions: keeping one thread alive
The fifth item on the list is the least technical and the one that changes the most, which is treating a conversation as a place rather than an event.
A thread accumulates. By message twenty it holds your product names, the tone you rejected twice, the customer segment you had to explain, the format you settled on. That accumulated context is worth more than any single prompt you could write, and the default behaviour of most people is to destroy it every few minutes by opening a new chat.
The practical version: keep one long running thread per recurring job. One for product copy. One for supplier correspondence. One for the monthly numbers. Go back to the right one rather than starting fresh, and when a thread gets slow or confused, ask it to summarise the working rules it has been following, then carry that summary into a new thread. You are moving the distilled context and leaving the noise behind, which is a manual version of what a memory feature tries to do automatically.
There is a limit to this and it is worth knowing. Every message you keep occupies room in the context window, so very long threads get expensive on paid tiers and eventually start losing detail from the middle. The signal is a model that suddenly forgets something you established early. When that happens, summarise and restart rather than fighting it.
What about the answers that are simply invented?
Assume any specific fact you did not supply may be wrong, and design your use around that rather than trying to prompt it away.
The pattern is consistent across everything published on this. Made up detail clusters around specifics: names, dates, citations, prices, regulation, product capabilities. It clusters much less around structure, tone, categorisation and rewriting, because those are judgements about text you gave it rather than retrieval of facts it might not hold.
That gives you a clean working rule. Tasks where you supply the source material are safe to trust at draft level. Tasks where the model supplies the facts are never safe to trust without a check, and the confidence of the answer tells you nothing, because the confidence is produced by the same process whether the fact is right or not.
The practical habit is to ask for sources you can open, then open them. If a link does not resolve or does not say what the answer claimed, you have learned something about that whole reply rather than about one sentence. Our note on checking whether a prompt change actually helped describes the same discipline applied to a repeated task rather than a one off answer.
When should a task leave the chat box?
When you have run it more than about once a week with substantially the same instructions. That is the crossover, and it is behavioural rather than technical.
The chat box wins while the job is still changing, because iteration costs nothing and you see every result. It starts losing once the job stops changing, for three reasons that compound. You are retyping stable instructions. You are copying results by hand into somewhere else. And you have no record of what you asked, so when the output drifts you cannot tell whether the model moved or your phrasing did.
Moving to the API fixes all three, at the cost of writing code and thinking about token bills. The prompt becomes a file you can version. The output can land directly in a sheet or a database. And the shape is enforced rather than requested. If you are weighing that jump, our comparison of how the main assistants differ once you get past the chat window covers what changes between them, and our note on prompt patterns with a before and after is the fastest way to see what a stable instruction looks like.
There is a third option people skip between chatting and coding: describe the tool you want and have it built for you. That is what MaShop does for commerce, generating a working site and admin from a description, with the code delivered to you rather than rented back. The store builder page explains how that flow works.
An hour that changes the ratio
Pick the task you did most often last month. Something dull with real inputs: tagging enquiries, writing variant descriptions, turning supplier emails into a comparison.
Do it once with the file attached instead of described. Correct the first draft three times in the same thread rather than restarting. Take whichever correction you made twice and move it into your standing instructions. Then ask for the result as named columns instead of prose. Four changes, one task, maybe forty minutes.
What usually happens is not that the output gets dramatically better. It is that it gets predictable, and predictable is the property that lets you hand a task over and stop watching it. That is the whole difference between using a chat assistant and having one do work for you, and none of it requires learning a single new term.
Two mistakes that survive every level of experience
The first is over specifying the output while under specifying the input. People write four hundred word prompts describing exactly what they want back, attached to two lines describing what they have. The ratio should usually be inverted. Give it a lot about the situation and comparatively little about the format, because the format is the part you can correct in one line afterwards and the situation is the part it cannot guess.
The second is treating a refusal or a bad answer as the end of the road rather than as information. When a reply comes back vague, the usual cause is a question with two answers in it. Split it. When a reply comes back hedged and full of caveats, the usual cause is a request for a judgement you have not given it grounds to make, so give it the grounds. Almost every disappointing answer traces to a missing constraint rather than to a missing capability, and finding which constraint is missing is faster than rewriting the whole prompt.
Neither of these is advanced. Both take one exchange to fix and both come up several times a week, which is the definition of a habit worth building deliberately rather than discovering slowly.