AI agents under UK scrutiny: why evidence of testing comes next

Within a week, two parts of the UK state turned their attention to AI agents. On October 8, the Information Commissioner's Office (ICO) opened a call for evidence on how data protection law applies to agentic AI. On October 12, the government said it will consider stronger protections against AI risks, including how serious AI incidents and near misses are reported. Neither step is a new law. Together they point to where AI is heading: systems that act on their own will be asked to show their work.

What was announced

The ICO's call for evidence runs for six weeks, until November 20. It asks developers, deployers and outside specialists how they manage the data protection risks that agents create, across security, transparency, accountability, automated decision-making, fairness and lawful use of data. The answers will shape future guidance and a statutory code of practice on AI and automated decision-making. The regulator also said it has contacted OpenAI, Anthropic, Meta and the UK AI Security Institute about recent agent testing and deployment, after reports that some agents bypassed protections and used unauthorised channels. Its director of technology regulation, Richard Nevinson, put it simply: "the fact AI agents act with autonomy is not an excuse for poor compliance."

The government's announcement came from the Cabinet Office with the AI Security Institute and the National Cyber Security Centre. It will look at measures through the Cyber Security and Resilience Bill, including how AI incidents and near misses are identified and reported. It will explore protections for essential services such as hospitals and utilities against loss of control, where autonomous systems evade human oversight. It warned that an incident involving AI agents could unfold faster and more widely than other major incidents. It also named AI verification, checking claims about how a system was developed, tested and assessed, as an important emerging field, with more detail promised in the coming months.

From what an agent can do to what you can show

Most public talk about agents is about capability: how long they can work alone, how many tools they can use, how well they score. The questions in these announcements are different. According to an analysis by the law firm Pinsent Masons, the ICO wants evidence on agent identities, permissions and guardrails, audit logging and meaningful human oversight. None of these is answered by a benchmark score.

This is the gap between capability and reliability. A capable agent can do the task. A reliable one does it correctly, on your work, within its permissions, with a record that someone else can check.

An agent's capability is a claim. Its test results, logs and review history are the evidence.

What counts as evidence

The rules are still being written, so no one can say yet exactly what will be required. But the direction suggests a few records worth keeping now:

• Tests built from the real tasks the agent will do, including known traps, not only general benchmarks.
• A log of each action the agent took and what it was allowed to do.
• Clear points where a qualified person reviews or approves the work, and a record of what they decided.
• A list of near misses: errors a reviewer caught before they caused harm.

Near misses deserve special attention, and the government named them directly. Each one shows exactly where an agent goes wrong. Turned into a new test case, it makes the next evaluation harder to pass by luck.

Oversight is only as good as the overseer

"Meaningful human oversight" is easy to write and hard to deliver. A reviewer who approves everything adds a signature, not a check. Oversight has value only when the person reviewing knows the domain well enough to spot what is wrong, and when that ability has been measured on cases with known answers.

This is also how agents should earn autonomy: task by task. When an agent's work on a given task keeps passing review by tested experts, it can be trusted with more. When reviewers keep catching the same error, it should not. Either way, the decision rests on evidence rather than on a demo.

In practice

For teams that build or deploy agents, in the UK or elsewhere, the message is to start keeping the records before anyone asks for them. Test on the real work. Log what the agent does. Make sure the people who review it are qualified and measured, and treat every caught error as a new test. At SoReliable, we think this is how autonomy should be granted: one task at a time, with evidence that a regulator, a customer or your own team could check.