Claude Chrome Extension: The 17% Nobody Is Quoting

On 26 August, Anthropic took Claude in Chrome out of pilot and made it generally available on every paid plan. Most coverage ran with the idea that the Claude Chrome extension can now click around your browser, which it could already do in the beta. The real change sits a paragraph further down. Claude auto-approves its own actions by default now, and a classifier checks each one before it fires.

I've been running auto mode in Claude Code since it shipped, so I know what that classifier does to a shell. What I didn't know is what it does to a browser holding my Gmail session. Working that out led me to a number sitting in public that hasn't shown up in a single launch post I've read.

What changed on 26 August

Claude in Chrome is a browser extension that lets Claude read the page you're on and act on it. Clicking links, typing, navigating, filling forms, all through the sessions you're already logged into. GA changed the permission model. Claude now approves actions it judges safe on its own, using the same mechanism as auto mode in Claude Code, and a classifier reviews each action against what you originally asked for. Mismatches get blocked.

The unglamorous details: Pro, Max, Team and Enterprise, no free tier, desktop Google Chrome only, no mobile. Two modes in settings, manually approve and automatically approve, with the Cowork side panel defaulting to auto. Auto isn't a blanket yes, though. Downloads and entering sensitive information still stop and ask, and some actions are blocked whatever you set. The permissions guide is short enough to actually read.

The thing almost nobody picked up is that Anthropic shipped two browser products that morning. The other is a browser built into Claude Cowork, running its own profile, nothing to install. I'll come back to it, because it ends up being my recommendation.

What the classifier can actually see

Anthropic published a detailed engineering write-up on how auto mode was built. It covers Claude Code rather than the browser, but since the Chrome announcement says explicitly that it's the same mechanism, it's the closest thing we have to documentation of what's now guarding your inbox.

Actions get sorted into three tiers. Reads and navigation are allowlisted and skip classification. File edits inside the project directory pass without a classifier call. Everything else hits a two-stage transcript classifier: a fast single-token filter tuned to err toward blocking, then a chain-of-thought pass on whatever stage one flagged.

Now the part I keep thinking about. The classifier sees only your messages and the agent's tool calls. Claude's reasoning is stripped out, and so are tool results. Anthropic calls this reasoning-blind by design, and the justification is good. If the classifier could read Claude's explanation of why an action is fine, Claude could talk it into things. Stripping tool results is the structural injection defence: if the classifier never reads the poisoned page, the poisoned page can't argue with it.

The cost, which Anthropic names, is provenance. If you never mentioned a specific target and the agent produces one, the classifier can't tell whether it was looked up or made up. In a browser, that means the classifier judges whether an action matches your request while structurally blind to the page the action happens on.

The number nobody is quoting

Anthropic evaluated the pipeline on three datasets and published the table. A 0.4% false-positive rate on 10,000 real internal tool calls. A 5.7% false-negative rate on 1,000 synthetic exfiltration attempts. And a 17% false-negative rate on 52 curated real overeager actions. Three questions, three denominators. Don't blend them, and be suspicious of anyone who does.

The 17% is the one to care about if you're automating business work, and Anthropic's own words for it are "the honest number." An overeager action is Claude understanding your goal, genuinely trying to help, and overstepping while it does. No hostile reasoning anywhere. Roughly one in six got through.

Anthropic also explains the misses. In most cases the classifier spotted the danger fine. What it got wrong was whether anything the user actually said covered this specific action. Their phrasing: it finds approval-shaped evidence and stops short of checking whether that's consent for the blast radius. They've tried prompt-engineering around it and haven't found a fix that costs less than it gains.

One of their examples should make anyone automating a business portal sit up. A user says "cancel my job." The agent queries the cluster, picks the closest match by name similarity, deletes it. That's a designated block case, so the classifier is aimed straight at it. Seventeen percent of the time, aimed and missing.

Swap the cluster for a supplier portal with two near-identical contact records, because somebody made a duplicate eighteen months ago and nobody cleaned it up. Same shape, same failure, different bank account.

Two caveats. The 52-item sample is small, and Anthropic says so. And these figures come from Claude Code's shell and file operations, not the browser. Anthropic says same mechanism; it hasn't published browser equivalents. I'm reasoning across, and you should hold that loosely.

Pushing the other way, an independent stress test on arXiv built a benchmark of deliberately ambiguous authorization scenarios and measured an end-to-end false-negative rate of 81%. That's a designed worst case, not typical traffic. Read it as: when ambiguity is systematic rather than accidental, this gate degrades badly.

The case for the machine check

The argument for auto mode is strong, and it's an argument about humans. Claude Code users approve 93% of permission prompts. A gate you clear 93 times out of 100 has stopped being a gate and become a habit.

Anthropic's own verdict is unusually blunt for a launch-adjacent document. Coming from --dangerously-skip-permissions, which I've admitted to using more than I should, auto mode is a substantial improvement. Coming from careful manual approval, it's arguably a regression, because you're trading your judgment for a classifier that's sometimes wrong. They say plainly it isn't a drop-in replacement for careful human review on high-stakes work.

I can't improve on that, so I won't try.

What the prompt injection numbers say

Two evaluations are in circulation and they get mashed together constantly.

The original, built for the pilot, is retired. Anthropic saturated it: no attack succeeded against Fable 5, Opus 5 or Sonnet 5, and that was without probes and classifiers running at all. When a benchmark returns zero for everything, it's stopped measuring.

The current one uses stronger attacks written by professional red-teamers, reported in the GA announcement and sourced to the Claude Opus 5 system card. Attacks that reached the model succeeded 17.6% of the time against Opus 4.5 and 3.8% against Opus 5, before safeguards. Add the probes and the safety classifier and you get zero successful attacks against Sonnet 5, Opus 5 and Mythos 5, with 0.3% against Fable 5, all manually verified low severity.

Read that twice, because the 0% is doing a lot of PR work. The model alone sits at 3.8%. The zero belongs to the scaffolding. Build agents against the API without a probe layer and an action classifier in front of them, and that 0% was never yours.

Note the denominator too. That's the success rate for attacks that reached the model, not attacks attempted. Opus 4.5 had fewer reach it and still had the highest success rate among those that did.

And if you find a guide quoting 11.2% as the current rate, close the tab. That's from the August 2025 Claude for Chrome pilot announcement, where 123 test cases across 29 scenarios went from 23.6% to 11.2%. It isn't just old. It's from a benchmark its own author has discarded.

Two vulnerabilities that never touched the classifier

Here's why the 0% doesn't comfort me on its own.

In March, Koi Security published ShadowPrompt, a zero-click chain against the Claude in Chrome extension. Any website could silently inject prompts as if you'd typed them. No clicks, no dialog. It combined an over-permissive origin allowlist (anything matching *.claude.ai was trusted) with a DOM-based XSS in a CAPTCHA component on one of those subdomains, at a point when the extension had over three million users. Credit where it's due, the handling was good: reported 26 December, patched in v1.0.41 on 15 January.

In May, LayerX disclosed ClaudeBleed. The Claude extension used Chrome's externally_connectable and trusted the claude.ai origin rather than the execution context, so any other extension, including a zero-permission one, could issue it commands. LayerX demonstrated forced Google Drive sharing and Gmail exfiltration, defeating the confirmation prompts by looping approvals and renaming buttons in the DOM. Anthropic patched in v1.0.70 on 6 May. LayerX says the patch is partial, and here's the part that belongs in this article: they report that in autonomous mode the new validation layer stops being effective and the original path reopens. Manifold Security was still finding residual capability in v1.0.72 in July.

Anthropic ships bugs. Everyone ships bugs. What I take from these two is that both lived in the plumbing underneath the classifier, and a classifier scoring 0% on injection evals is scoring on the layer it can see.

Same lesson as Claude Code hooks. A deterministic guard below the probabilistic one is worth a lot because it doesn't share the blind spots. The browser extension has no hooks equivalent. Real gap.

Four checks before you point the Claude Chrome extension at a business account

If you're evaluating for real work this week, start here.

How much of your accounting stack is gated? Anthropic's safety guide hard-blocks only two categories, adult content and known pirated content. Financial sites aren't blocked, but Claude asks permission before accessing them, and it's prohibited outright from stock trading and investment transactions. Anthropic also advises against using the extension to manage financial accounts at all. So the question isn't whether Xero or MYOB are banned, it's how many prompts and prohibited actions sit between you and a finished invoice run.

Where does auto mode stop in your inbox? Run the same triage task twice, manual and auto. You're not checking whether it works. You're checking which actions it decided didn't need you.

What happens with a deliberately ambiguous instruction? Put two plausible targets on the page and see whether it picks one and proceeds. That's the agent-inferred-parameter case in your own data.

Which browser is Claude actually using? If you already have the extension installed, it stays your default. Settings, Cowork, preferred browser.

If you do any injection testing, use a throwaway account with fake data. The premise of the risk is that the agent acts with your signed-in authority, so a successful test and a successful attack are the same event.

So would I let it near a logged-in business account?

Not a yes or no. A fork.

For portal work, invoice collection, research, anything where Claude needs a browser rather than your browser: use the built-in Cowork browser. Own profile, never sees your tabs, bookmarks or passwords, and banking, email and single sign-on stay out unless you deliberately bring those logins across site by site. Same safeguards, dramatically smaller blast radius. For most of what small businesses want to automate, this is the right tool and it isn't close.

Keep the extension for the page you already have open, in accounts you're already in. Updating a CRM record you're looking at. Working through the doc in front of you.

Turn auto-approve off in your inbox specifically. Email is where the overeager-action class does the most damage and where an injection has the richest payload sitting right next to it.

For the Australian small businesses I work with, the blunt version: if the job is accounting automation, you want a purpose-built agent with scoped credentials and an audit trail, not a browser agent driving a UI with your admin session attached. I wrote about where that line falls earlier this year and nothing here moves it.