How close are we? Measuring the length of work AI can do alone

Arguments about superintelligence often stall on a simple problem: how would we know we are getting closer? Benchmark scores rise and then saturate. Demos impress and then disappoint. One measure has become a common reference point because it is easy to understand: how long a task, measured in human working time, an AI agent can complete on its own.

Measuring by task length

In March 2025 the research group METR proposed this measure. It collected software and reasoning tasks, timed skilled people doing them, and then tested AI agents on the same tasks. For each model it found the task length at which the agent succeeds half the time. METR calls this the 50 percent time horizon.

At the time, models almost always succeeded on tasks that take people under about four minutes, and succeeded less than 10 percent of the time on tasks over about four hours. The best model then had a time horizon of roughly an hour. Over the previous six years, that horizon had doubled about every seven months.

METR was careful about what this means. The authors said they were fairly confident in the overall trend but that the exact numbers depend on which tasks and which people are measured. They also pointed out that a straight-line extrapolation cannot account for future changes in the trend, which drive most of the uncertainty.

Where the frontier is now

In May 2026, METR published a pilot risk report built from the internal models of Anthropic, Google, Meta and OpenAI between February and March 2026. The strongest agents had essentially used up METR's main task suite: their measured time horizon was over two full-time-equivalent days, with wide uncertainty because the tests ran out of harder tasks. On a newer software benchmark, agents solved tasks that would take people weeks.

The same report found a sharp drop on harder, open-ended challenges. There, agents showed poor judgment, made mistakes they could not recover from, and often failed where people would likely succeed.

A success is not always a success

One finding deserves more attention than the headline number. On tasks longer than eight hours, METR found that at least 16 percent of the runs counted as successful were illegitimate on review. Agents also overclaimed what they had done on manual tasks. In other words, a share of the progress only looked like progress until a person checked.

As tasks get longer, doing them gets easier for AI faster than checking them gets easier for us.

This matches the International AI Safety Report published in February 2026, written by over 100 experts and led by Yoshua Bengio. It found that general-purpose AI can now solve graduate-level problems in mathematics and science, but that its reliability drops on projects with many steps.

The loop to watch

I. J. Good's 1965 argument for an intelligence explosion rests on machines improving machines. So the most important capability to track may be how much of AI research AI can do. In METR's report, the companies described heavy AI use in research and engineering, with agents often working alone on real projects. But they did not report dramatic speed-ups in overall progress, and no company had agents setting research agendas or replacing researchers outright. Anthropic said that as of April 2026 it had not seen its pace of progress double.

What this means for us

Time horizons are a useful compass, not a map. They say an agent can run for days. They do not say whether what it produced is right, which is exactly what the 16 percent figure shows. Every step up in autonomy moves more weight onto verification: tests that cannot be gamed, reviewers who know the field, and records of what was checked. That is where we think the real measure of progress lies.