AI agents simulated a supply chain and lost to humans, not even close

The Beer Game is a supply-chain simulation built at MIT in the 1960s. Four stations sit in a line: a retailer, a wholesaler, a distributor, and a factory. Beer flows downstream toward the customer, and orders flow upstream toward the factory. Each station sees only its own desk, meaning its own inventory, the order that just arrived from below, and the shipment that just landed from above. Nobody talks to anyone else. Every week each station decides one number: how many cases to order from its supplier. Orders and shipments both take time to arrive, so a decision made today does not land for weeks.

The game has one famous lesson. A small, one-time change in customer demand, passed up a chain of people who cannot see past their own station, turns into wild swings in orders and inventory. The retailer nudges its order up. The wholesaler, seeing that nudge on top of its own thinning shelf, orders a bit more. The distributor more again. By the time the signal reaches the factory, a change of a few cases has become an order for dozens. This is the bullwhip effect, and it is expensive: every unit held in inventory costs money each week, and every unit a customer wants but cannot get costs more.

We ran the game with AI. We asked Claude Opus 4.8, the flagship model at the time of writing, to design and run the experiment, and it did what frontier models now do with a large task: it broke the work into pieces and handed each piece to a separate sub-agent. The workflow it wrote spins up sixteen agents, one per station across four teams, and gives each one only its own station's view through the same interface a human player uses. Those sub-agents ran on a smaller, faster model. None of them could see another station, another team, or one another. They could not coordinate. Every week, each agent read its own numbers and placed its own order.

That structure is the point. Decompose a goal, distribute the parts to a fleet of agents that each work in isolation, collect the results: this is how state-of-the-art AI increasingly operates. So this is not a toy about a board game. It is a clean look at what happens when that pattern meets a problem where the parts depend on one another. We compared the outcome to human teams who played the identical game in person, and to the cost of a perfectly coordinated chain.

Cost ran away from the ideal

Through the first ten weeks every team, human and AI, tracked the ideal line closely. The chain starts in balance, and the demand step had not yet worked its way upstream. Then the AI teams separated. From around week 12 their cumulative cost climbed steeply and kept climbing. The human teams bent only slightly above the ideal and stayed there.

The four AI teams finished between 4,787 and 7,160 in whole-chain cost over 36 weeks, a mean of 5,852. The human teams finished at 1,564 and 2,064 over 32 weeks. The AI cost did not spread evenly across the chain. It piled up in the wholesaler and distributor seats in the middle, where each agent reacted to a swing that was already amplified below it, then amplified it again. The retailers, closest to the steady real demand, stayed cheap.

Orders amplified up the chain

The bullwhip shows clearest in the orders themselves. Customer demand never rose above 8 units. Yet the further an AI agent sat from the customer, the larger its biggest order grew. The human orders barely grew up the chain at all.

Averaged across teams, the AI retailer's largest order was 16 units, the wholesaler's 32, the distributor's 54, and the factory's 59. The worst AI factory ordered 80 units in a single week, ten times true demand. On the same measure the human stations sat at 9, 13.5, 16, and 18, and the worst human factory peaked at 20.

Watching one chain week by week shows the mechanism in motion. In Crimson, the most expensive AI team, the retailer's orders stayed near real demand the whole game, while the distributor and factory swung between near zero and 80 and back, chasing a backlog that their own earlier over-ordering had created.

The swings left mountains of stock. The AI distributor seat peaked at 150 units of inventory on one team while running deep backlogs earlier in the game, the classic overshoot-then-overcorrect cycle. No human station ever held more than 39 units.

Same game, a fraction of the cost

Because the human game ran 32 weeks and the AI game 36, we read each AI team's cost at week 32 and set it beside the humans' final cost. The gap holds.

At week 32 the AI teams averaged 4,709 in whole-chain cost. The humans averaged 1,814. A perfectly coordinated chain would have cost 1,472. The humans landed 23 percent above the ideal. The AI teams landed 220 percent above it, playing the same game under the same rule that nobody may talk.

What this means

The agents were not short on capability. Each one could read its station and do the arithmetic. What they lacked was a shared view and any feedback loop to the rest of the chain. Starting context, rules, and demand were identical to the humans'. The difference was the missing connection between roles. Sitting alone with a local signal, every agent treated a temporary backlog as a reason to order more, and that extra order became the next station's shock.

Why did the humans, under the same no-talking rule, stay calmer? Here we can only offer a hypothesis. Even without speaking, people in a shared room read a slower pace, a facilitator's rhythm, and each other. The agents had none of that. Reacting quickly to the number in front of them, in complete isolation, they did not damp the signal. They amplified it. This is an interpretation of why the cohorts differed, not a measured cause.

The AI had more context, not less

Were the agents underinformed? Not at all. To build the experiment, the orchestrating model read the simulation's entire codebase. It understood every mechanic in detail: the two-week shipping and ordering delays, the exact cost of holding versus backordering, the single demand step at week five. Human players were given the rules of a board game. The model was given the source code.

So context was not the human advantage. What several of the humans had instead was experience. For some of them, this was not their first time at the table. Experience is not the same as context. Context is what you are told before you start. Experience is the judgment you reach for in the moment, especially when things go wrong. A player who has already watched a backlog spiral once does not need to be told to hold their order. They feel it coming, and they wait.

Context engineering is useful, but it has its limits. The agents knew the rules cold. But they had never been burned by them.

A local maximum that looks like progress

Step back and this is a local-versus-global maximum problem. Each agent climbed its own hill: clear my backlog, refill my shelf, cover the order in front of me. Every one of those moves was locally sensible, and locally it looked like progress. More orders placed, more units in motion, more inventory built. At nearly every station the AI chain was busier than the human chain. It was also three times further from where the whole chain should have been, the entire time.

That is the uncomfortable part. The failure did not look like failure. It looked like output. A station buried in inventory and a station ordering eighty units against a demand of eight are both, locally, doing something. The motion is real. The units are real. The activity reads as performance. The only thing missing is the one thing that would reveal the problem, which is a view of the whole.

It is worth asking what happens as this pattern spreads. As teams lean harder on AI to produce and decide, the first sign is not less output. It is more: more drafts, more tickets, more analyses, more decisions, each one defensible on its own. The quiet cost is the shared context that used to keep everyone aimed at the same global target. This experiment cannot prove that organizations are drifting this way. It can only show, in a controlled setting, how easily a fleet of capable, isolated optimizers ends up far from the global best while every local reading says things are fine. Call it being blinded by the performance of it all: so much visible motion that nobody stops to check the map.

There is a second cost worth naming. Playing a game that people run with pen, paper, and poker chips took the agents 592 separate decisions and about 13.6 million tokens, across 1,769 tool calls and 37 rounds. Isolation is expensive twice over: once in the inventory bill, and again in the compute spent generating it.

Limitations

• One run. A single session of four AI teams against two human teams, not a controlled study. The numbers describe what happened here, not a population.
• Different lengths. The AI game ran 36 weeks and the human game 32. Every cross-cohort comparison uses the matched 32-week figures or states the length.
• Naive agent policy. The agents ran on a simple prompt and a small fast model, with no memory of strategy between weeks and no tuning. A different prompt or a reasoning model could behave very differently.
• Experienced human players. Some of the human team had played the Beer Game before, so this is seasoned humans against first-attempt agents. We treat that experience as part of the result, not a flaw to control away.
• Small, non-random cohort. Four AI teams and two human teams. The human players were not sampled at random.
• A simulation, not an operation. Real chains carry prices, variable lead times, and forecasts the game leaves out.