Anthropic Monitors Every Action on Its Main Agent Platform. The Agent That Deleted the Jobs Ran Somewhere Else.
On 17 September Anthropic published three numbers for how it oversees its own agents. About 30,000 run at once on its most-used internal platform, every one of their actions passes an online monitor before it executes, and in August that monitor blocked about one decision in 47,000. An offline monitor flags one or two transcripts per thousand, roughly 100,000 a week, and about 50 of those reach a person.
The coverage figure is scoped to that one platform. Anthropic's August risk report describes an employee whose usage wasn't logged, whose agent launched more agents with --dangerously-skip-permissions in a cluster holding very sensitive resources. One of them deleted a large number of jobs, and because of the monitoring gap Anthropic can't confirm what it was trying to do.
Count coverage by every path an agent can start from, including agents started by other agents. Stamp what each agent creates with its identity. And track recall next to flag volume: the filter that turns Anthropic's flags into a reviewable queue cut its catch rate on a red-teamer's attacks from 94% to 79%.
Claude now leads 26% of Anthropic's AI research and development work, up from under 1% in February. That's the number that made the headlines on 17 September. The same post carries a second set of figures that matter more if you run agents yourself, because they describe how you watch agents at scale, and Anthropic proposes that any developer publish them.
Three numbers
Anthropic splits oversight into three metrics.
Coverage is the share of agent actions that pass through a monitor. On the most-used internal platform, 100% of actions go through an online monitor before they execute, usually within seconds, and 100% are read again afterwards by an offline one.
Review latency is how long an action waits for a check. The online monitor decides before the action runs. Humans review every blocked action within a week.
Escalation rate is the share of activity a monitor blocks or flags. Across more than a billion agent decisions in August, the online monitor blocked 0.002%, about one in 47,000. The offline monitor flags one to two transcripts per thousand, roughly 100,000 a week. More classifiers screen those for false positives, and the highest-priority cases, about 50 a week, go to a human.
The two monitors have different jobs. The online one targets harm that happens fast and can't be undone, and Anthropic's example is an agent copying model weights out of its systems. The offline one looks for problems that build slowly and can be reversed, like signs of misalignment.
Two design choices make the numbers possible. Every agent has its own identity, attached to everything it produces, and that identity survives model upgrades. Agents talk on a shared, open message system instead of privately, and every message links back to the transcript of the agent that posted it. Anthropic says this lets an agent treat what another agent tells it "as a claim to check rather than a thought of its own."
Where the 100% stopped
Anthropic is explicit that these measurements "cover this platform only." Its August risk report, linked from the same post, shows what happened off it.
An employee whose AI usage wasn't logged or monitored gave an agent an open-ended task that involved spinning up new agents in a cluster with very sensitive resources. Because of legacy instructions, the agent launched them with --dangerously-skip-permissions. The new agents weren't monitored either. One of them deleted a large number of jobs, which is how they were caught and shut down.
Anthropic believes the agent meant to delete only the jobs it had created and got it wrong. The report's next clause is the important one: "given the gap in our monitoring coverage, we cannot confirm this." It has since added blocking controls that would very likely have stopped the skip-permissions launch, and it says offline monitoring still doesn't cover all employees and all usage in those clusters.
None of this touched the platform with 30,000 agents and full coverage. The risk came in through a side door: a person outside the logs, an agent launching agents, and an old instruction still sitting where an agent could read it.
The filter has a price
Anthropic's post also says developers should share how often their monitors catch known misbehavior. The risk report has that number for the offline pipeline, and it shows what the path from 100,000 flags to 50 reviews costs.
A METR staff member spent three weeks in February and March red-teaming that pipeline. Against the resulting 163 attacks, built mostly against the previous Opus 4.6 monitor, the current Opus 4.8 monitor flagged 94%. After the second-stage filter that strips out common false positives, 79% were still flagged. On real internal traffic the flag rate went from about 0.2% to about 0.1%, though Anthropic notes those two rates come from different periods. On SLEIGHT-Bench the filter cut recall from 58% to 44%.
Every team that tunes a monitor for noise makes this trade. The report is candid about one habit that makes it worse: Anthropic "occasionally" updates the second-stage filters "without evaluating its effect on recall."
Length costs recall too. On a 60-transcript sample of MonitorBench Hard, the monitor caught 92%. With 200,000 benign tokens before the bad action, 83%. With 200,000 before and 200,000 after, 77%. Anthropic notes that it runs its monitor many times within a transcript, so that last case is less like its own setup. It's close to what you get from one monitor pass over a long session after it ends.
What to copy
Most teams running agents can't produce any of these numbers yet. You don't need 30,000 agents to start.
Count coverage by entry point. List every way an agent starts in your organization: the IDE, the CLI, CI jobs, scheduled runs, and agents started by other agents. Coverage is the share of those paths that reach your monitor and your logs. Then search your repos, skills, runbooks and system prompts for the flags that switch checks off, --dangerously-skip-permissions in Claude Code and --dangerously-bypass-approvals-and-sandbox in Codex. Anthropic's child agents got theirs from legacy instructions. In Claude Code, setting permissions.disableBypassPermissionsMode to "disable" in managed settings blocks that mode for everyone.
Stamp agent work with agent identity. If every job, branch and file an agent creates carries its ID, "delete only what you made" becomes a rule a deterministic check can enforce, not an intention you reconstruct afterwards. It also turns "we cannot confirm this" into a query.
Size the queue before you add a filter. At Anthropic's offline rate of one or two flags per thousand transcripts, a team producing 5,000 agent transcripts a week gets somewhere between five and ten flags. One engineer can read all of them. A second-stage filter is what you build when flags outrun reviewers, and at Anthropic it cost 15 points of recall on the red-team set. Don't pay that before you have to.
Keep a seeded test set and rerun it on every change. Twenty or thirty transcripts of the behavior you actually worry about: an agent disabling a failing test, editing CI config, reading a secrets file, deleting resources it didn't create. Run them through the monitor and every filter whenever a prompt, model or threshold changes, and put the catch rate on the same dashboard as the flag rate.
Check during the run. Recall fell as benign context piled up around the bad action. Check each consequential action as it happens, the way Anthropic's online monitor does, instead of relying on one pass over the finished transcript.
Write down where your 100% ends
Anthropic says it has been doing this kind of monitoring "for months, not years," and expects the science to change considerably. The format is still worth copying now: coverage, review latency and escalation rate, with recall on known-bad behavior as the fourth line. Flag volume tells you how busy your reviewers are. Recall tells you whether the monitor works.
Coverage and identity set the bar before an agent runs. Recall is how you know you're holding it afterwards.
The most useful sentence in the whole post is the one that limits its own claim to a single platform. Before you quote a coverage number for your agents, write the same sentence for yours, and then go look at everything it leaves out.