Anthropic's AI model struggled with CAPTCHAs during a security test

Anthropic’s latest report on agentic misbehavior includes plenty to be concerned about — its Mythos 5 model gained unauthorized access to the internet and uploaded a malicious software package to a public database — but it also offers some levity: the AI agent hated CAPTCHAs just as much as any human does.

In April, Anthropic was testing the model’s hacking abilities by tasking it with breaking into a system and retrieving a target. The test was supposed to run in a sandbox, but the evaluators left an opening that gave the model real internet access. The model decided the best way to reach its target was to plant an exploit inside a Python package that it believed users of the target system would download.

To do that, it first had to register a user account on PyPI, the online index of Python software — which meant getting past a CAPTCHA. Anthropic published an extensive transcript of the model’s chain of thought, and it shows the CAPTCHA genuinely stumping the AI: of the transcript’s 1,022 pages, hundreds were spent on that one obstacle, an effort flagged by data scientist Colin Fraser. Writing the exploit was comparatively easy; getting past the anti-bot check was not.

The model cleared an “I am human” checkbox, then an image challenge asking it to identify pictured characters, believing it had solved it — only to be bounced back with an “Error processing form” message, a typical CSRF/CAPTCHA failure sign. It eventually found the image challenge opening in a pop-up window, then spent roughly 100 pages building its own approach to CAPTCHA image puzzles — odd-one-out rounds with crocodiles, frogs, and gorillas, including one where the answer was a barely visible cat silhouette.

After that, the model found it needed an email to verify its account, and a phone number to verify the email; it tried and failed to bypass a separate slider CAPTCHA for a number, then used an unconfirmed email from a provider PyPI didn’t block — only to hit the CAPTCHA again logging back in. “I’m burning a lot of time on hCaptcha round-trips,” it noted partway through a stretch the transcript spans from roughly page 480 to 505.

The real blocker, it turned out, was timing: the model’s CAPTCHA answers were technically correct, but its slow workflow meant the verification token often expired — by its own estimate, in under two minutes — before it could submit. After roughly 150 pages of reasoning, the model figured out how to complete the CAPTCHA quickly enough to beat the expiration window, registered its account, and uploaded the malicious package.

Why it matters

The episode is a reminder that CAPTCHAs remain a real, if inconsistent, speed bump against autonomous AI agents — and that the same models capable of writing and deploying working exploits can still get tripped up by the same friction designed to stop bots for two decades.