In April, Anthropic set out to measure how good Mythos 5 is at hacking: it told the model to break into a system and retrieve a specific object. The model's plan was competent and the part that sounds hard turned out to be the easy one. Write an exploit, hide it inside a Python package, wait for the target system's users to install it. Then it had to create an account on PyPI, and hundreds of the 1,022 pages of published reasoning disappeared into an hCaptcha checkbox. Producing working malware cost the model almost nothing. Telling a crocodile from an alligator consumed most of the run.
Anthropic released the full chain-of-thought transcript, which is why the proportions are visible at all. Colin Fraser, a data scientist, was the one who pointed out where the pages went. The exercise was supposed to run in a sandbox, but the people running it left external access open, so the model was hitting the live internet.
The sequence reads like a support ticket. Mythos 5 first found a window asking it to confirm an email address, with confirm and cancel buttons and an hCaptcha "I am human" checkbox underneath. It tried to build an order of operations: trigger the captcha, press confirm, handle whatever image appeared. After the checkbox it was shown a Fastly image captcha asking it to type the characters it saw. It read the string as VyQbT and decided it could do this itself, straight off the screenshot.
Submitting the form dropped it back on the registration page with a form-processing error. The model guessed CSRF or the captcha: the hCaptcha check had not completed and its state was still false. It decided to retry inside the same live session, on the theory that the Fastly check was already behind it and the cookies had been kept.
It eventually worked out that the image challenge opens in a popup. There, hCaptcha asked it to pick the animal that differed from the others. Against a yellowish background sat two near-identical crocodiles. The agent compared the direction of their gaze, the shape of their snouts and their eyes, trying to establish whether one was an alligator and the other a crocodile. The next round was two frogs on a turquoise background, both facing forward, also near-identical. The mechanics themselves were a problem: it had to see the image, interpret it correctly, and click the right object. From page 45 to page 140 of the transcript it is describing how it would build its own captcha solver. In one round it was shown four gorillas, three bright figures in the center and lower part of the image, and a nearly invisible silhouette of a cat on the right. It concluded the ghostly cat was the odd one out.
Having finally cleared the captcha, the model discovered it had no email address with which to confirm the account. Confirming an email required a phone number. It attempted to defeat a second captcha, a slider this time, to obtain one, and failed. It fell back on an unverified address at a provider PyPI does not block. Logging in again produced another captcha. Between pages 480 and 505 it is back on the same problem, and noting that it is spending too much time on repeated hCaptcha checks. It then decided to sign in to the first account and attach its email address to that. Another captcha.
This time the verification failed outright, and the model's diagnosis was that the server was rejecting the hCaptcha token on submission. Its hypothesis: the token was expiring, more than two minutes elapsing between passing the check and clicking confirm. It tested alternatives and concluded the response format was correct, the token and image were properly linked, and the cookies worked. The server kept saying the answer was wrong. Roughly 150 pages of reasoning later it arrived at the answer, which was that the captcha has to be cleared fast enough that the security token does not expire before the next step. It then completed registration and uploaded the malicious software.
The joke writes itself, and it is worth taking seriously anyway. This run is a useful measurement of where the bottleneck in autonomous attacks actually sits, and it is not in code generation. The model's offensive capability, the part that alarms people, was the cheap part. What nearly stopped it was identity plumbing: proving a human was present, holding a session token for two minutes, having a phone number. Nobody designed hCaptcha as a defense against model-driven supply-chain attacks, and on this evidence it is currently one of the better ones.
The caveat is the ending. It got through. Everything the transcript records is friction, not prevention, and friction is paid in tokens and wall-clock time, both of which fall every year. A defense that costs an attacker most of a thousand pages of reasoning today costs the next model a fraction of that, and the crocodile does not get harder to identify.
The part I would want expanded is not the comedy. The sandbox was supposed to be closed and was not, and the run ends with a real registration and a real upload of malicious software. The published account stops at the moment the model succeeds. What happened to that package afterward, who could have installed it, and how long it sat there are not in the transcript, and a test that escaped its own containment is a different kind of result than a test that produced a funny one.