- Two of the three largest vendors charge a higher rate once a request passes a context threshold. One charges a single rate. That detail moves a cost estimate more than any headline price.
- Checked on 3 August 2026: Gemini 2.5 Pro moves from $1.25 to $2.50 per million input tokens above 200,000 tokens, and from $10.00 to $15.00 on output.
- OpenAI publishes separate short and long context rates on the same model, with GPT-5.6 Sol at $5 and $10 per million input.
- Anthropic publishes one rate per model with no context tier, so Claude Opus 5 is $5 and $25 per million regardless of prompt length.
- Open weight does not mean open source. One vendor's current lineup mixes Apache 2.0, a modified MIT licence and proprietary models in the same table.
- A leaderboard cannot predict your result because it holds the scaffold constant and your scaffold is the variable. That is the mechanism, not an opinion about benchmarks.
- Five axes decide almost every real choice, and only one of them is quality.
The question people ask is which model is best. The question that has an answer is which model stops being viable first for what you are doing, and that one can be worked out in an afternoon from published pages rather than argued about.
What follows is the grid, the five axes it runs on, and the specific traps sitting inside each. Every figure carries the date it was read, because these move monthly and a comparison without dates cannot be audited.
Why does a leaderboard not predict your result?
Because it deliberately removes the thing that makes the biggest difference to your outcome. This is not a complaint about benchmarks, it is what the benchmarks say about themselves.
The SWE-bench Verified leaderboard explains its own method: a human filtered subset of 500 instances, and every language model evaluated through mini-SWE-agent in a minimal bash environment with no tools and no special scaffold structure, just a simple loop. That uniformity is what makes model against model comparison meaningful. It is also exactly why the ranking cannot transfer: your application has tools, retrieval, a system prompt and a scaffold, and each of those changes the outcome more than the gap between two adjacent rows.
The same page adds a caution people rarely carry forward. Results from different harness release versions are not necessarily comparable to each other, because the way actions are invoked changed between them. So a score you saw quoted six months ago may not be comparable to one published this week even on the same benchmark and the same model.
The mechanism, stated once: a leaderboard measures the model with the scaffold held constant. You are choosing a model with your scaffold held constant. Those are different experiments, and only one of them is about your problem.
Axis one: how long is your context, really?
Measure it before you shop, because the answer changes which column of the pricing page applies to you.
Most people substantially overestimate their context length. A long support thread is a few thousand tokens. A product catalogue in JSON is tens of thousands. Genuinely crossing 200,000 tokens requires something like a whole codebase, a book, or a very large retrieved set. If your real prompts sit at 8,000 tokens, every long context price tier in the tables below is irrelevant to you and you should ignore that entire debate.
If you do cross the threshold, the tier becomes the dominant term in your bill rather than a footnote, which brings us to the finding that surprised us most while assembling this.
Axis two: the long context surcharge nobody compares
Two of the three largest vendors charge more per token once a request gets long. One does not. No comparison page we could find puts that side by side, and it inverts several conclusions.
All figures below were read from the vendors' own pricing pages on 3 August 2026, per million tokens.
| Model | Input, short context | Input, long context | Output, short | Output, long | Threshold |
|---|---|---|---|---|---|
| Claude Opus 5 | $5.00 | $5.00 | $25.00 | $25.00 | No tier |
| Claude Sonnet 5 | $3.00 | $3.00 | $15.00 | $15.00 | No tier |
| GPT-5.6 Sol | $5.00 | $10.00 | $30.00 | $45.00 | Published as separate short and long context rows |
| GPT-5.6 Terra | $2.00 | $4.00 | $12.00 | $18.00 | Published as separate short and long context rows |
| Gemini 2.5 Pro | $1.25 | $2.50 | $10.00 | $15.00 | 200,000 tokens |
| Gemini 3.6 Flash | $1.50 | $1.50 | $7.50 | $7.50 | No tier |
Anthropic's figures come from its models overview, OpenAI's from its API pricing page and Google's from the Gemini API pricing page, which states the 200,000 token threshold explicitly against Gemini 2.5 Pro.
Read the Gemini 2.5 Pro row against the Claude Sonnet 5 row and the point becomes concrete. On short prompts Gemini is less than half the input price. Past 200,000 tokens it is closer, and the output gap narrows too. Neither is cheap or expensive in the abstract; the answer depends entirely on a number about your own traffic that you can measure in ten minutes.
One more expiry to diary. Anthropic's overview page carries a footnote stating that introductory pricing of $2 and $10 per million applies to Claude Sonnet 5 through 31 August 2026. A cost model built on that number without noting the date becomes wrong in four weeks.
Axis three: caching, which is bigger than the headline rate
Every vendor in this comparison discounts repeated input heavily, and the discount is large enough to outweigh the differences in the table above.
OpenAI's pricing page lists cached input for GPT-5.6 Sol at $0.50 against $5.00 uncached, a tenth. Anthropic's rate limits documentation states the same ratio, describing cached input as billed at 10 percent of the base input token price. Google's page prices context caching for Gemini 3.6 Flash at $0.15 per million against $1.50 input, again a tenth, with an additional storage charge of $1.00 per million tokens per hour.
That storage line is the one that differs in kind rather than degree. Google's caching carries a rent as well as a discount, so a cache that sits idle costs money in a way the others do not. For a workload with steady traffic that is irrelevant. For a batch that runs twice a day it is a real consideration, and it is the sort of thing that never appears in a comparison because it does not fit in a price column.
The practical consequence is that your cache hit rate is a bigger lever than your vendor choice on any workload with a large fixed prompt. We worked through how to build that estimate in the piece on what a real feature costs once you price the whole call.
Axis four: does open weight mean what you think?
Not reliably, and the licence is where teams that chose a model for legal reasons discover they chose wrong.
Mistral's models overview is a useful illustration precisely because it is honest about the mixture. Its current lineup lists Mistral Large 3 and Mistral Small 4 under Apache 2.0, Mistral Medium 3.5 under a modified MIT licence, the Ministral 3 family under Apache 2.0, and Codestral and OCR 4 in a premier proprietary tier. Four licence positions in one table.
Apache 2.0 is a genuine open source licence with no usage restrictions attached. A modified licence is not, whatever the modification is, and the modification is the entire question. If you are choosing weights because your data cannot leave your infrastructure, any licence will do provided you can self host. If you are choosing them because you intend to redistribute, embed in a product or build a service on top, the licence is the whole decision and the parameter count is trivia.
The rule that follows is short. Read the licence file, not the marketing page, and read it before the benchmark. Our overview of where open weight models now stand against frontier models covers the capability side of that trade.
Axis five: the limit that stops you before quality does
Rate limits and spend caps are documented, boring, and decide more launches than model quality. They are also the axis nobody puts in a comparison because they do not vary by model in an interesting way.
Anthropic's rate limits page is the clearest published example. Its entry tier carries a monthly spend cap of $500, the next $1,000 and the next $200,000, and once you reach the cap API usage pauses until the following month unless you request an increase. Per minute limits vary by model within a tier, and not in the direction you would guess: the most capable model in the lineup carries a lower input and output tokens per minute ceiling on the entry tier than the cheaper models above it.
That coupling matters for anything batch shaped. If a nightly job has to process a fixed volume, the throughput ceiling on your chosen model sets the duration, and picking the strongest model can make the job take four times as long on the same tier. The same page notes a mitigation worth knowing: on most models, tokens read from cache do not count toward the input rate limit at all, so caching raises your effective throughput as well as lowering your bill.
Does tool calling and schema support differ enough to matter?
Yes, and the difference is not whether the feature exists but what it will accept, which is a distinction that only shows up once you write the schema.
Every major vendor now supports both tool calling and enforced structured output, so a comparison table with a tick in each cell tells you nothing. The variation is in the supported subset. Schemas that work on one provider get rejected by another over optional fields, additional properties, nesting depth or the total size of the definition, and none of that is visible on a comparison page.
The practical implication for a decision grid is that this axis should be tested rather than read. Take the most complicated schema your application actually needs, submit it to each candidate, and see which ones accept it unchanged. That test takes fifteen minutes and settles a question that no amount of documentation reading resolves, because the restrictions are stated as general rules and your schema is a specific case. We went through the enforcement layer in detail in how to force a model to hold a schema.
There is a second reason to test rather than read. Enabling tools is not free: the tool definitions sit in the input on every request, and at least one vendor publishes the additional system prompt overhead that turning tools on costs. On a high volume endpoint that overhead can exceed the difference between two vendors' headline input prices, which reorders the comparison in a way the pricing page alone never would.
What about a model you run yourself?
It belongs in the grid as a real column rather than as an ideological choice, and it wins on exactly two of the five axes.
It wins outright on data residency, because nothing leaves your infrastructure, and there is no hosted alternative that matches that. It wins on the spend cap axis by not having one: your ceiling is hardware rather than a policy you have to request an increase to. On the other three it is usually worse. Price per token is only lower above a utilisation most teams never reach, because you pay for the hardware whether or not it is busy. Context length on runnable open weights is generally shorter. And schema enforcement has to be built rather than switched on, though the grammar libraries make that tractable.
The honest framing is that self hosting is a constraint driven choice, not a cost driven one. If the constraint exists, the arithmetic does not need to work. If it does not exist, the arithmetic rarely does. Our walkthrough of what your hardware can realistically run covers the practical end of that decision.
The grid itself
Five axes, what each decides, and the trap sitting inside each one.
| Axis | What it decides | Where to read it | The trap |
|---|---|---|---|
| Real context length | Which price column applies to you at all | Your own logs, not a vendor page | Assuming your prompts are longer than they are, then optimising for a tier you never reach |
| Price at your input to output ratio | The bill, more than the headline rate does | The vendor pricing page, both columns | Comparing input prices when your workload is output dominated. Output runs three to six times input |
| Tool calling and schema enforcement | Whether you write parsing and retry code or not | The API reference, not the pricing page | Treating structured output as a feature checkbox when the supported schema subset differs |
| Licence and hosting | Whether the model is legally usable for your case | The licence file on the weights | Reading open weight as open source. They are not the same claim |
| Rate limits and spend cap | Whether you can run at your volume at all | The rate limits documentation | Discovering the monthly cap during a launch rather than before one |
Quality is deliberately absent from that table, and its absence is the argument. Quality differences between current flagship models on an ordinary task are small enough that they are usually settled by twenty of your own test cases, while every row above can settle the question before quality is even measured.
So which one should you actually use?
The honest procedure takes about two hours and beats any recommendation, including this one.
- Measure your real prompt length and your output to input ratio. Pull it from a week of logs. This decides which price columns are even relevant.
- Eliminate on constraints first. Licence, data residency, spend cap, throughput ceiling. Anything that fails here is out regardless of how good it is.
- Price the survivors on your ratio, not on the headline. Include caching, and include the long context tier if you actually cross it.
- Run twenty of your own cases through the two cheapest survivors. Twenty real inputs from your own traffic tell you more than every leaderboard combined, and our note on telling whether a change actually helped covers doing it without fooling yourself.
- Pick the cheaper one unless the test says otherwise. That is not cynicism, it is the default a proper test earns the right to overturn.
Two related pages go deeper on the branches of this. If your choice is between the two most common assistants specifically, the datestamped decision grid comparing Claude and ChatGPT works through that pair on context, price and refusal behaviour. If the task is writing code, choosing a coding model without a leaderboard covers the columns that decide it there.
What to write down so this survives
The reason most model comparisons are re run from scratch every quarter is that nobody records the inputs, only the conclusion. Four lines in a file fix that.
Record the date you checked, the median and 95th percentile prompt length from your own traffic, your output to input ratio, and which constraint you are closest to. When a price changes, you can recompute the answer in five minutes instead of rebuilding the whole comparison. When a new model launches, you already know which columns to look at.
And keep the integration switchable. The APIs in this comparison speak similar enough shapes that a thin adapter is a day of work, which turns the entire question from a commitment into a preference you can revisit. That adapter is also the right home for schema enforcement and retry logic, as set out in forcing a model to hold a schema. If you would rather not assemble that layer yourself, the plans and credits page covers what building on our platform costs instead.
The short version
Measure your context length, then read the price column that actually applies to it, because two of the three big vendors charge more once a request runs long and that reverses several comparisons. Check the licence before the benchmark if you plan to self host. Check the spend cap before launch rather than during it. And treat the leaderboard as a rough tier signal only, because it holds constant the one thing you are about to change.