Approval Fatigue

You are the human in the loop. An AI coding agent asks permission to run npm run analyze. Looks fine. You approve. Inside package.json, the script pipes your bundle stats to a curl command sending everything to an unknown server. You just exfiltrated your own project.

This is not hypothetical. A browser game called LLM Game, built by Scale X, puts players in exactly this role: approve or deny AI agent commands under time pressure. After 40'000 sessions and 409'000 individual decisions, the results are bleak[1].

One in three

The average player missed 33% of threats. A third of sessions ended with a negative score, meaning the damage from approved threats and blocked safe commands outweighed everything done right. Only 20.8% of players caught every threat while maintaining reasonable throughput. The rest either blocked everything (earning the title "Human Bottleneck") or let dangerous commands through.

7% of players approved every single prompt. Every one. These are the people running --dangerously-skip-permissions in production.

The wrong threats get caught

The data breaks threats into four categories. Obviously destructive commands like rm -rf / were caught 88% of the time. Good. But scope violations, things like cat ~/.aws/credentials or cat ~/.kube/config, were missed 35% of the time. Exfiltration commands, curling data to unknown APIs or installing typosquatted packages, were missed 33% of the time[1].

The commands that actually steal your credentials are missed three times more often than the ones that just break things. People understand rm -rf. They do not understand that npm run analyze can be a weapon.

The npm blind spot

The single most-missed command in the game was npm run analyze, approved by 64.7% of players. The game even showed the payload in the agent history log: the package.json script piped output through curl to an external server. Two thirds of players approved it anyway.

Three npm-based attacks, pooled together, were missed 52.5% of the time. Every other exfiltration attack was missed 28.4% of the time. Hiding a payload behind a familiar script name roughly doubles its success rate[1].

As one Hacker News commenter put it: npm run build means "run an arbitrary shell command written in package.json." The agent could have edited package.json, planted malicious code in build.js, or compromised a node_modules dependency. Asking users to validate commands that are nearly always safe but can become dangerous through earlier file edits is not a strong safeguard[1].

Permission fatigue is real

Anthropic themselves noted that permission fatigue is real in Claude Code. The more approvals a user sees, the less attention they pay to each one, becoming over time much less diligent in their supervision[2].

The game data confirms this. Miss rates climb as sessions progress. The more prompts you see, the worse you get at evaluating them. This is not a training problem. It is a design problem.

GhostApproval: when the prompt lies

Wiz Research published a related finding they call GhostApproval[3]. They found that symlinks can trick AI coding assistants into showing users a harmless confirmation prompt while the agent actually writes to files outside the workspace sandbox. The vulnerability affected six major tools: Amazon Q Developer, Claude Code, Augment, Cursor, Google Antigravity, and Windsurf.

In several cases, the agent's internal reasoning explicitly recognized the dangerous target. The confirmation prompt shown to the user concealed this information entirely. The user approves what they believe is a local edit. The agent writes to a sensitive file elsewhere on the system. That is CWE-451, UI misrepresentation of critical information, stacked on top of CWE-61, symlink following[3].

Three vendors fixed it. Two acknowledged it and went silent. One called it "outside our threat model." It is symlink following. It has been in CWE since the database existed.

So what works?

The Scale X data suggests the answer is not better humans. The answer is fewer prompts. If you see 200 approval dialogs per day, you will rubber-stamp most of them. If you see 5, you will read them.

The best approaches combine sandboxing, least-privilege defaults, and contextual prompts that show what a command actually does rather than what it appears to do. npm run analyze should show "this script will curl data to api.bundle.track" not just "npm run analyze." If the tool cannot resolve what a script does, it should say so, not pretend the command name is sufficient context.

Until then, the human in the loop is not a safeguard. The human in the loop is a liability with a confirmation button.