01
Long horizon
Changes ripple through traffic, services, budgets, and population over simulated time.
AI agents × live simulation
Can an AI
run a city?
CitiesBench tests whether an agent can improve life in a functioning 100,000-person city—not just solve a puzzle in one move.
Bridge verified · evaluation in development
Public seed · smoke capture
100K+
Population
75%
Traffic flow
Live
Simulation
~100K
resident seed city
Native
Windows simulation
Fixed
time + action budget
Private
post-run evaluation
Why CitiesBench
Intelligence that has to live with its decisions.
02
Partial observability
The agent must inspect the city, form a hypothesis, act, and measure what changed.
03
Real consequences
Bad interventions can strand residents, collapse services, or erase prior gains.
04
Comparable runs
Frozen starts, fixed budgets, transcripts, and post-run evaluation make agents comparable.
One controlled run
From checkpoint to evidence.
The model sees the objective and game state. It never sees evaluator code, weights, or comparisons to the hidden baseline.
- 01
Load
Every official run begins from the same frozen checkpoint.
- 02
Observe
The agent reads city state through a constrained game bridge.
- 03
Operate
It changes infrastructure and policy within fixed budgets.
- 04
Evaluate
A hidden evaluator measures the city after the agent exits.
Integrity by design
The test stays hidden. The evidence does not.
Evaluator code and private checkpoints remain outside the agent sandbox. Official runs publish configuration commitments, complete action traces, and reproducible artifacts.
Read the integrity protocolBuild status
Preview✓Authenticated game bridge and controlled stepping
✓Screenshots, tool traces, and direct model loop
·Frozen private checkpoint and full QoL tool surface
·Committed evaluator and first official model runs
Protocol preview