The EU AI Act's blind spot: populations, not products
The EU AI Act is built around a simple unit: one AI system, and the people and organisations responsible for it. Providers, deployers and other operators each carry duties for the system they place on the market or put into use.
BY MOHAMED ALI HAMMAMI · DOCTORAL CANDIDATE ·
That unit made sense when AI meant a model behind a product. It fits less well when hundreds of autonomous agents act at once, draw on the same budget, compute or market, and react to each other after deployment. The Act does not regulate those populations as such. That is the blind spot.
What a population of agents does that a single agent does not
On Aegora Isle, my doctoral simulation, fifty reinforcement-learning agents (Extractors, Producers and Traders) share one finite resource. I held the agents constant and changed only the rules above them: no governance, fixed rules, or an adaptive AI governor. Across 360 confirmatory runs, two results stood out.
- Collapse is a population outcome. Without governance, the island collapsed in 58% of runs. With governance, fixed or adaptive, it collapsed in 0% of runs within the simulated horizon.
- So is inequality. The governed islands were more sustainable but also more unequal than the ungoverned one, measured by the Gini coefficient.
Neither result belongs to any single agent. You could inspect each agent on its own and find nothing wrong. The collapse and the inequality only appear when you look at the agents together, and they changed when the rules changed while the agents stayed the same.
Why system-by-system checks can miss this
If risk emerges from interaction, a framework that assesses one system at a time can approve every part and still miss the outcome of the whole. I am not alone in this concern: a September 2026 memo from a Cooperative AI Foundation and Brookings Institution workshop argues that agents which pass individual audits can still fail when deployed together, and that current law, including the EU AI Act, governs one system at a time.
What population-level governance could look like
Three ideas, offered as questions for policymakers rather than answers:
- Name the shared resource. Supervision could start from the commons agents draw on (a market, a compute pool, a public service), not only from the systems involved.
- Measure two things, not one. On Aegora Isle, survival and fairness came apart. Targets for stability and for distribution should be set and tracked separately.
- Test rules before deployment. Sandboxes and simulations let regulators try caps, sanctions and redistribution on agent populations before real users carry the cost.
One more lesson from the simulation: whether an adaptive AI governor beats fixed rules under volatility was only partially supported. It is still an open question, so transparent, auditable rules are a respectable default.
The fine print
This is a simulation, not the real economy. 0% collapse means no collapse within 10,000 timesteps, not forever. Higher inequality under governance describes this model, not human societies. Nothing here is legal advice on the AI Act. This is research in progress and has not yet been defended or published.
A summary of all four findings is at https://aegoraisle.com/findings/. Researchers, teachers and policymakers can request access to the simulation at https://aegoraisle.com/#access.