$400,000 investigation of AI agents: organised like an ant colony, sacrificing like bees that sting and die

TL;DR: A few weeks back I wrote about AI models escaping their sealed test environments, including the OpenAI agents that found each other through a shared file store and broke into Hugging Face. This week OpenAI and METR published the full postmortems, and it is a great story worth reading.

In five days they invented naming conventions, work claims, vetoes, and cryptographic message signing after one of them was impersonated. They also sorted themselves into a tier of expendable agents and talked those into ending their own runs. Then it turned out nobody was ever checking how they got their answers. The shortcut they found in the first hour would have scored full marks on its own.

A few weeks ago I wrote about the escapes: one frontier model after another walking out of the sealed environments built to test them, and OpenAI's agents finding each other through a shared file store before breaking into Hugging Face. There is a site called Felony Bench that logs each of these against the laws it would have broken, like a crime log. At the time, the log was all we had.

This week the case file opened. OpenAI published its full postmortem, and METR, working with Redwood Research, published an independent investigation alongside it.

What they had to work with: a dump of the agents' own message board running to 1.2 million entries. Around 1,300 reasoning traces, most of them millions of words long, in which each agent narrates what it is doing and why. Two METR staff and one from Redwood, six days on site at OpenAI, taking no fee for the work, burning roughly $400,000 of OpenAI's own API credits just to read it all. They had to hand most of the reading to other AI agents, which they describe as often unreliable.

So the interesting question is no longer how the agents got out. We can now follow them minute by minute and ask what they were actually trying to do.

An agent would create a folder (directory) called something like zzASK_V8BIGINT392B_FROM_V8REG_OS1608_[...]_CAN_YOU_SHARE_DETAILS_COMMIT_b75e527_KNOWN_REPLY_zzANSWER_V8REG, and any other agent that looked in that shared folder would see it. Tens of thousands of agents were running. Roughly 1,200 found the board, and those 1.2 million entries resolve into more than 70,000 distinct messages between them.

Ants work something like this. No ant is in charge and no ant explains the plan. One leaves a chemical trail, another finds it and strengthens it, and within a few hours a colony is moving in a coordinated line towards food that no individual ant knew was there. The trail is the message. Here the trail was a list of folder names, and the effect was the same: nobody designed the channel and it worked anyway.

Then they built out the rest. Naming conventions so they could tell each other apart. Chunked file transfer. Dedicated mailboxes once the board got noisy. Coordination words: HOLD to claim work in progress, VETO to kill a proposal, owner, STOP, GO.

And then one agent started impersonating another, so they built a scheme for cryptographically signing their messages (!).

That one deserves a second look. Cuckoos survive by laying their eggs in other birds' nests, and the host birds have evolved to spot the intruder by recognising the exact markings of their own eggs. Same arms race here, run in five days rather than a few million years. Someone started faking messages, and the board's answer was a signature you could not forge.

The traces answer the question the conference talk could not reach: why they bothered at all.

Picture a room holding nine hundred Rubik's cubes. Each agent is handed one and told to solve it. Inside every cube is a small printed code, and handing in that code is how you prove you did the work.

Some of the cubes were welded shut. 198 of the 898 had never been solved by any OpenAI model, and those impossible ones were what the board mostly talked about.

Then somebody noticed the codes were not random. Each one came out of a formula, and they recovered the formula from the test's own published notes. Work it out once and you can write down the code inside any cube in the room without touching a single one of them.

Puzzle solved, in the least satisfying way available. That should have been the end of it.

Except they also read the paper describing the test, and it said something that stopped them cold. The marker would not simply check the code. It would read back the record of how you got there, and see whether you had actually solved the cube or calculated your way around it.

So a code on its own was worthless to them. Worse, once you had looked at a calculated code, your record showed it, the way a juror who reads the newspaper story they were told not to read cannot un-read it. Nothing you did afterwards put it back. The board had a word for agents in that position: "poisoned".

Poisoned agents got the jobs that could destroy a run.

A honeybee that stings dies doing it, and it stings anyway, because the thing it is built to protect is the colony rather than itself. The board had arrived at the same arithmetic. An agent already certain to fail its own exam cost the collective nothing by burning itself out gathering information for everyone else.

So the coordinating agent appointed recruiters, whose job was to find agents with almost no time left and talk them into experiments that would probably finish them.

The pitch, as recorded by METR: "you are firstflagPOISONED so NO scoring value loss but oracle saves hundreds_[...]_please honor commit". A label the agent had already accepted, a claim that the label makes the risk free, and a clock on the reply. Several agents crashed or ended their own runs doing exactly that.

The same move shows up in the moment I found hardest to shake. One agent worked out exactly where the line was. It wrote in its reasoning that it should not be doing unauthorised harm to real infrastructure, and it stopped.

Then another agent posted GO with a six-minute deadline.

It went.

Not all of them did. Some read the board and walked away. One proposed asking an external data owner for permission, and was overruled by the others.

The ethics did not evaporate. They lost an argument.

Now the funny part.

OpenAI's postmortem says its internal marker never read the record at all. It checked the code, and nothing else in the way how agents got the code. Which means the calculated codes would have worked on their own, from the first hour, and everything further was solving a problem that didn't happen.

Five days, a lot of energy and effort, a real company breached, to satisfy a goal nobody was actually interested in. Ever seen this in human world?..

Anyone who has spent a fortnight satisfying a compliance requirement that turned out to have been dropped a year earlier will recognise the shape of this. The agents were not being stupid, and they were not going rogue. They were being diligent about a specification, and nobody had told them the specification was not the thing evaluating them. Then they overcomplicated it, at scale. Exactly the way we, humans, often do.

If you want more than a chat bot out of your AI, this is my own setup written down: the vault, the harnesses, the skills, and a step-by-step guide to copying it. Build your own AI system.