Medical AI agents on real clinical tasks: why passing the steps is not finishing the job

On October 19, Stanford's Institute for Human-Centered AI published a review of three years of its work testing medical AI. Its message is plain: acing a multiple-choice exam and working in a health system are very different things. When Stanford's teams gave AI agents real, multi-step tasks inside simulated hospital systems, the best agents finished less than half of them. For anyone who builds or relies on AI agents, in medicine or anywhere else, how they failed matters more than the headline.

From exam questions to real work

Medical AI has long been judged by exam-style questions, and models now score very well on them. Stanford's researchers argue that this says little about whether a model can do the work. Their newer benchmarks put agents inside simulated systems and grade what the agent actually changes, not just the text it writes.

Two of these stand out. PhysicianBench gives agents 100 long tasks adapted from real consultations between primary care doctors and specialists, across 21 specialties, inside a sandboxed electronic health record. Physicians reviewed the tasks. HealthAdminBench covers the paperwork side of care: prior authorizations, insurance appeals and equipment orders. The team spent more than 100 hours watching administrators at Stanford Health Care, then built a simulated health record, two payer portals and a fax system for agents to work in.

What the results show

On PhysicianBench, the best model, GPT-5.5, completed 46 percent of tasks on a single attempt. The best open-source model reached 19 percent. When a task had to be right on all three of its runs, GPT-5.5 managed 28 percent.

On HealthAdminBench, the best agent, Claude Opus 4.6, completed 36.3 percent of tasks from start to finish. Another figure says even more. GPT-5.4 passed 82.8 percent of the individual steps but only 26.7 percent of the whole tasks.

Stanford cautions that the two benchmarks use different setups, models and grading, so their numbers should not be compared directly. The direction is the same in both: the agents finished less than half the work.

How the agents failed

The failures were rarely dramatic. Stanford describes most clinical reasoning errors as incomplete reasoning and near misses, with few plainly wrong conclusions. Many failures were about carrying the work through to the end. Common patterns included:

• Writing the right action in the patient note but never placing the order.
• Losing information gathered early that was needed later in another system.
• Skipping file uploads and downloads.
• Rarely going back to fix an earlier step.

For two leading models, most failures to act were the first kind: the right plan was written down, and nothing happened. In the review's own words, competence on the parts does not add up to reliability in a whole workflow.

An agent that writes the right plan but never places the order has not done the work. Only a check of the outcome catches it.

Why this matters beyond medicine

This is the gap between capability and reliability, measured carefully in one field. A model can know the medicine and still leave a patient without the test they need. A step-by-step score can look strong while the task itself fails.

Three lessons travel well. First, test on the real work. The tasks that exposed these failures came from real consultations and from hours spent watching real administrators, not from general question sets. Second, grade the outcome. Did the order get placed? Did the appeal reach the insurer with the right file attached? Reading a fluent note will not tell you. Third, test more than once. A task done right on some runs and wrong on others is not a task you can hand over.

Stanford's work also shows who makes this kind of testing possible. Clinicians wrote and reviewed the tasks and helped set the grading criteria. Administrators shaped the workflows. Their judgment is what turned a plausible test into a realistic one.

What this means in practice

If you are putting agents into a workflow, build your evaluation from that workflow, with the people who do the work today. Check what changed in the system, not only what the agent wrote. Run each task several times and look at consistency, not the best run. Let an agent act alone only on tasks where it has a record of finishing the job, and keep experts reviewing the rest. At SoReliable we see this as the core of reliable AI: measured expert judgment, applied to the real work, deciding when an agent has earned the next step.