- There is one genuinely useful number for AI agents: the length of task, in human time, that an agent finishes with 50 percent reliability. METR calls it the time horizon.
- That horizon has been doubling roughly every seven months since 2019, and a frontier model measured at around 50 minutes in early 2025 was measured at over two hours by GPT-5.
- The horizon is measured on software, machine learning and cybersecurity tasks, against a low context human baseline, and METR states measurements above 16 hours are unreliable with its current task suite.
- The best explanation of why long jobs fail is a constant failure rate per minute of work, which gives every agent a half life and makes success fall away exponentially with task length.
- The practical consequence is a design rule rather than a purchase: shorten any single unsupervised run, and put a checkpoint where a person can see the work.
An agent that completes a 50 minute job half the time will complete a four hour job almost never. Not a bit less often. Almost never, because the failures multiply rather than add. That single piece of arithmetic is the most useful thing to know about AI agents, and it is missing from nearly every page that explains what they are.
This piece is built around two numbers and one model. The numbers come from the only serious attempt to measure agent capability in units a person can use. The model explains why the numbers behave the way they do. Together they let you decide what to hand to an agent without reading a benchmark table again.
What is an AI agent, as distinct from a chatbot with tools?
An agent chooses its own next step. That is the whole distinction, and everything else is marketing. A chatbot with tools answers you and may call a function you set up. An agent is given an outcome, decides what to do first, looks at the result, and decides again, repeatedly, without being asked each time.
The word covers five quite different levels of autonomy, and knowing which rung you have bought explains what will go wrong. Most products sold as agents sit on rungs two and three.
| Rung | What the system decides | What the person decides | What failure looks like | Example |
|---|---|---|---|---|
| 1. Single answer | Only the wording of the reply | The question, and what to do about the answer | A wrong answer you can see immediately | Asking a model to draft a product description |
| 2. One tool call | Whether to call the tool and with what arguments | Which tools exist, and the order of the work | A badly formed call, or the right call on the wrong record | A model looking up an order status to answer a customer |
| 3. Scripted chain | The content at each step | The sequence of steps, written in advance | A step silently producing rubbish that the next step accepts | Draft the email, translate it, then queue it for sending |
| 4. Self directed within a toolset | Which tools, in what order, how many times, when to stop | The goal, the tool list, and the limits | Looping, giving up early, or declaring success on a half done job | An agent asked to reconcile a week of invoices |
| 5. Open ended, can spawn or browse | All of the above, plus creating sub tasks and acting on the open web | Only the goal and the permissions | Compounding errors nobody watched accumulate | An agent researching suppliers and filling in forms |
The rungs matter commercially because the failure mode changes character. On rungs one and two you see the mistake. On four and five the system reports completion and the mistake is somewhere in the middle of work nobody watched. That is also where the authority question starts to bite, which we looked at separately in what an AI agent can commit your business to.
How long a task can an AI agent actually finish?
Long enough to be useful and much shorter than the pitch implies, and for once there is a real measurement. METR defines the 50 percent time horizon as the task duration, measured by how long a human expert takes, at which an agent is predicted to succeed half the time. Its task completion time horizons page, last updated 8 September 2026, publishes those measurements for frontier models.
The original paper, Measuring AI Ability to Complete Long Tasks, reported Claude 3.7 Sonnet at a 50 percent time horizon of around 50 minutes, and found the frontier horizon doubling approximately every seven months since 2019, with a possible acceleration in 2024. The authors attribute the trend mainly to greater reliability and a better ability to adapt to mistakes rather than to raw reasoning. On the measurements page, a GPT-5 agent is given a horizon of around 2 hours and 17 minutes.
Before carrying that anywhere, carry its limits with it, because METR states them plainly and they decide how far the number travels.
- The tasks are software, machine learning and cybersecurity work. They are self contained, well specified and automatically scorable. METR notes in follow up work that agent performance drops substantially when scored holistically rather than algorithmically.
- The human baseline is low context. METR says a two hour task is better understood as what someone with little or no prior context, like a new hire or a freelance contractor, could do in two hours, not what an experienced professional does in their own codebase.
- Above 16 hours the measurement stops meaning much. The page carries that warning directly: measurements above 16 hours are unreliable with the current task suite.
- A horizon is not a promise about any single task. On tasks taking a human 90 minutes to 3 hours, METR reports a GPT-5 agent succeeding every time on about a third, failing every time on about a third, and being inconsistent on the rest.
That last point deserves a moment. The average conceals three populations: work the agent reliably does, work it reliably cannot, and work where it is a coin toss. If you are deciding what to automate, your job is to find out which third your task is in, and no published number can tell you that.
Why does success fall away so fast as jobs get longer?
Because the failure rate is roughly constant per minute of work, so survival decays exponentially. This is the model that makes the whole subject legible, and it comes from Toby Ord's paper Is there a half life for the success rates of AI agents?, which showed that agent performance on the METR suite is well explained by a constant rate of failing during each minute a human would take to do the task.
If that holds, every agent has a half life: the task duration at which its success probability hits 50 percent, which is exactly what the time horizon measures. Double the task length and you do not lose a little accuracy, you halve the already halved chance. Ord's own interpretation of why it fits is that longer tasks involve increasingly large sets of subtasks where failing any one fails the whole thing. He is careful to add that whether the model applies to other task suites is unknown and needs more work.
The independent evidence points the same way. A 2026 synthesis of 27 benchmark, taxonomy and audit papers spanning 19 benchmarks reports that failures compound nonlinearly with task length, that strong performance on individual subtasks does not reliably translate into end to end success, and, importantly for anyone buying a wrapper product, that additional scaffolding does not consistently improve reliability.
Three things follow immediately, and they are worth more than any model comparison. An agent that aces a subtask can still fail the job, so demos of steps prove little. Letting an agent run longer makes things worse rather than better, because length is the thing that kills it. And the engineering response is not a better model, it is a shorter horizon per unsupervised run.
What are the six ways agents actually fail?
The 2026 synthesis groups them into six clusters. Each one has a symptom you would notice in a small business, and a response that does not require understanding the internals.
| Failure cluster | What you would see | What to do about it |
|---|---|---|
| Tool invocation and parameter errors | The right action taken on the wrong record, or a call that quietly does nothing | Make destructive actions require confirmation; log every call with its arguments |
| Planning and constraint satisfaction | A plan that ignores a rule you stated, like a budget or a blackout date | Enforce constraints in the tool, not in the prompt; the tool refuses, the prompt only asks |
| Long horizon degradation from context accumulation | Coherent early, drifting later, contradicting its own earlier steps | Shorten runs; start fresh rather than continuing a long session |
| Multi agent coordination | Duplicated work, contradictory outputs, a handoff that drops half the task | Prefer one agent unless the work genuinely splits into independent parts |
| Safety and security under adversarial or vague conditions | Following instructions embedded in content it read | Treat anything fetched as untrusted; keep credentials out of reach |
| Measurement validity | Benchmark scores that do not match what you observe | Measure on your own work with your own pass criteria |
The fourth row is the one people pay for by mistake, and the published cost figures back the caution: our piece on what agent teams actually buy for the extra spend found team setups costing between 1.8 and 5.1 times a single agent while winning one comparison in four.
What is a horizon budget, and how do you set one?
A horizon budget is the longest you let a single unsupervised run go before a person or a hard check looks at the output. It is the one design decision that follows directly from the half life model, and almost nobody writes it down.
Setting it takes three steps and no technical knowledge. First, estimate how long the task would take a competent person with no special context, since that is the unit the horizon is measured in. Second, compare that to what the agent reliably manages on your work, which you learn by running it twenty times on real cases and counting. Third, cut the work into chunks shorter than that figure, with something checkable at the end of each.
The checkpoint has to be a real one. A summary the agent writes about its own work is not a check, for reasons the self reporting evidence makes uncomfortably clear. A real checkpoint is a diff you can read, a draft that will not send until approved, a reconciliation that balances or does not. Our write up of what a support agent's resolution rate actually counts is the same argument applied to one metric.
How do you measure an agent on your own work?
In an afternoon, with twenty runs and a notebook. This is the part that replaces reading benchmarks, and it is the only evidence that applies to your business.
Pick one real task you do repeatedly and that somebody could check. Write the pass criterion first, in one sentence, before you see any output: the total matches the statement, or the reply contains the delivery date and the returns window. Then collect twenty real cases from the last month, including the awkward ones, because the awkward ones are where the money is lost.
Run all twenty. Score each as pass or fail against the criterion you wrote, with no partial credit, and keep the failures. Then read them. You are looking for which of METR's three populations your task sits in: reliable success, reliable failure, or a coin toss. Fourteen out of twenty is not a 70 percent success rate you can deploy, it is a strong hint that two distinguishable kinds of case are hiding inside one task, and the useful move is usually to split them.
Time the human version while you are at it, roughly, as somebody with no special context would do it. That gives you the unit the published horizon is measured in, so you can sanity check your result against the trend rather than against a vendor claim. If your task takes a person twenty minutes and the agent manages it eighteen times out of twenty, you are comfortably inside the horizon. If it takes four hours and the agent manages six, the answer is not a better model.
Which jobs are the right shape for an agent?
Short, verifiable and reversible, in that order of importance. The shape of the task matters more than the quality of the model, and you can judge shape without any benchmarks at all.
A good candidate has a success criterion you could hand to someone else. Did the invoice total match the statement. Did the draft include the three facts required. Did the stock count reconcile. A bad candidate is a job whose completion is a matter of judgement, because there is nothing to check against and the agent will report success either way.
Reversibility is the second filter. An agent that drafts something a person sends is a different risk from an agent that sends. An agent that proposes a refund is different from one that issues it. The published failure clusters above include acting on instructions found in content it read, which is a reason to keep the irreversible step behind a human hand even when the agent is good.
There is one structural point from METR worth borrowing here, because it clarifies what a long task even is. METR notes that a thousand separate one hour problems is not a thousand hour task, it is a one hour task done a thousand times, since the work splits across parallel workers. Long means the steps depend on each other: debugging where each fix reveals the next problem. Volume is cheap to automate. Dependency is what breaks agents, and most small business work that looks big is actually volume.
What does the tool layer have to do with reliability?
More than the model choice does, on two of the six failure clusters. An agent acts only through the tools it is given, so the tools are where limits actually live. A constraint written into a prompt is a request. The same constraint enforced by a tool that refuses is a rule.
This is the practical reading of the planning and constraint cluster. If an agent must not refund more than a set amount, the refund tool caps it. If it must not email outside business hours, the send tool queues instead. Prompt instructions get followed most of the time, which is a different thing from being followed, and long runs are exactly where most of the time stops being enough.
It is also why the interface standard matters commercially rather than just technically. Tools exposed through a documented protocol can be given narrow permissions, logged per call and revoked individually, which is what makes a rung four agent auditable instead of mysterious. Our own MCP server exists for that reason: a defined surface an agent may touch, with everything else closed. Whatever stack you use, the question to ask a vendor is which actions are enforced by the tool and which are merely requested in a prompt.
The browsing case deserves its own line, because it crosses two clusters at once. An agent that reads the open web is reading untrusted text, and text can contain instructions. That is the safety cluster meeting the tool cluster, and it is the single strongest argument for keeping the irreversible action behind a person. We traced the seller side of that exposure in what a shop controls when a browser agent arrives.
Does more scaffolding or a bigger setup fix this?
Not reliably, on the published evidence, and that is the finding most likely to save you money. The synthesis of 19 benchmarks states directly that additional scaffolding does not consistently improve reliability.
Parallel setups are a partial exception, with a specific condition attached. Anthropic's engineering write up on building a multi agent research system reports that on one browsing benchmark token usage by itself explained 80 percent of the variance in performance, and that multi agent systems use about 15 times the tokens of a chat interaction. Their conclusion is explicit about the condition: these architectures need tasks valuable enough to pay for the increased performance, and they suit heavy parallel search across more material than one context window holds. They also say most coding tasks are a poor fit, because the pieces depend on each other.
Which is the half life model again, from the other direction. Work that splits into independent pieces can be thrown at many agents, because each piece is short. Work whose steps depend on each other cannot, because you are back to one long chain where a single failure ends the task.
The measurements on this page, and when they were read
This page is built to be re measured rather than rewritten, so the figures carry dates. The time horizon definition and the model measurements come from METR's time horizons page, read in October 2026 and itself last updated 8 September 2026, where that page notes it is no longer actively updated. The doubling of roughly seven months since 2019, and the 50 minute figure for Claude 3.7 Sonnet, come from the March 2025 paper. The GPT-5 horizon of about 2 hours 17 minutes, and the breakdown of outcomes on 90 minute to 3 hour tasks, come from the measurements page. The 16 hour reliability ceiling is stated there too.
The half life model is from Toby Ord's May 2025 paper, which fits the METR research engineering suite and states that generalisation to other suites is unknown. The six failure clusters, the nonlinear compounding with task length and the finding on scaffolding come from the 2026 synthesis of 27 papers across 19 benchmarks. The token ratios and the parallel search condition come from Anthropic's engineering write up of its own multi agent system.
What would change this page: a new horizon measurement on a current model, a replication of the half life result on a non engineering task suite, or published evidence that scaffolding reliably improves end to end reliability. The structure of the argument, which is that length is the enemy and checkpoints are the answer, survives all three.
What to expect next, and how to re read this page
The trend line is the most load bearing claim here, so treat it carefully. A doubling every seven months is a historical fit, not a law, and METR argues an exponential fits its data better than linear or hyperbolic alternatives while noting it has seen no evidence of slowing. Extrapolating it years out is the reader's risk, not the measurement's.
Two practical implications if the trend continues. The set of tasks inside the horizon widens steadily, so a job that failed twenty times last year is worth retesting rather than assumed impossible. And the half life structure does not change with the horizon: longer runs will still decay faster than shorter ones, so the design rule outlives any particular model.
One caution on reading any agent number, including the ones here. The measurements page itself stopped being actively updated in September 2026, and METR is clear that its coverage was never a complete record of the most capable models. So the right use of a time horizon is as an order of magnitude and a direction of travel, not as a specification. Your own twenty runs on your own work beat any published figure for deciding what to automate this month.
The short version
An agent is a system that picks its own next step, and the honest measure of one is the length of job it finishes half the time. That length has been doubling every seven months or so and currently sits in hours on clean software tasks, measured against a low context human. Success decays exponentially with task length because the failure rate per minute is roughly constant, which means every agent has a half life and no amount of extra agents or scaffolding reliably changes that. So the work is not to pick a better model. It is to cut the job into pieces shorter than the horizon, give each piece a check somebody can read, and keep the irreversible step in human hands.