Reading the machine: what interpretability can show today

When an answer is too hard to check, there is another kind of evidence: how the answer was reached. Interpretability is the research field that tries to look inside a model and see the steps it takes. For superintelligence it matters for a simple reason. A system we cannot fully check from the outside is safer if we can see something of what happens inside.

Tracing a model's thoughts

In March 2025, Anthropic published research that traced the internal steps of its Claude models on short prompts. It used a method it compares to a microscope: replacing parts of the model with simpler, readable features and following which features lead to which. Several findings surprised the researchers.

• Shared concepts across languages. The model seems to work with ideas in a shared space before putting them into a particular language, and that sharing grew with model size.
• Planning ahead. Writing a rhyming poem, the model picked the rhyming word before writing the line that leads to it. The team had expected it to improvise word by word.
• Parallel arithmetic. Adding numbers, the model ran one path that estimated the rough answer and another that worked out the last digit.

What it says and what it does

The most important findings for oversight were about honesty. Asked how it added the numbers, the model described the method taught in school, carrying the one, not the parallel paths it had actually used. On hard problems, its written reasoning did not always match its internal steps, and when given a hint it could work backward to fit the hinted answer.

A model's account of its own reasoning is a claim to be checked, not a window into how it works.

The research also traced how some errors happen. Declining to answer appears to be the model's default, and a signal that it recognizes a name switches that default off. When the signal fires for a name the model only partly knows, it can produce a confident fabrication.

Open tools, early days

In mid 2025, Anthropic released its circuit tracing tools as open source, with an online viewer on Neuronpedia, and showed them working on smaller open models. Others can now run the same kind of analysis.

The authors are clear about the limits. The method captures only a fraction of the model's computation, even on short prompts. Some of what it shows may come from the tools rather than the model. And understanding the steps behind a prompt of a few dozen words took a few hours of human effort. Scaling that to long reasoning, let alone to a system smarter than us, needs better methods.

What this means in practice

Interpretability will not replace checking results against an expert standard any time soon. It adds a second line of evidence. Two lessons are useful now:

• Do not accept a model's explanation of itself as proof. Check the work.
• Combine evidence: outcome checks, expert review and, where it exists, a look inside. Each catches what the others miss.