Oracle, Genie, and Sovereign AI Roles
Classify AI by its societal role before assessing utility and risk
- Difficulty
- Moderate
- Time to result
- ~days to results
- Steps
- 6
- Confidence
- 99%
Classify an AI by the role it is designed to play. An oracle answers questions but does not act in the world. A genie executes a bounded task and returns for more instructions. A sovereign pursues an open-ended, long-range objective through many self-selected subtasks. This classification concerns external function, not internal architecture: an oracle may use multiple research agents, while a sovereign could be a single unified agent. Risk grows differently across the roles. Oracles can provide dangerous information; genies can carry out harmful actions; sovereigns can become powerful optimizing forces that steer the future toward a flawed mission. Every type can also contain unintended goal-seeking processes, so review both role-specific misuse and hidden internal agency.
Origin
Nick Bostrom presents oracle, genie, and sovereign as three AI templates with different utility and safety tradeoffs.
Core principles
- 01Classify the role the system plays, not its internal architecture
- 02Match autonomy to the breadth and duration of the objective
- 03Assess intended behavior and emergent internal goals separately
- 04Trade utility against the risk surface of each role
How to run it
- 1
Describe the external role
State what the system is expected to do for users and when it returns control.
Pro tip Use observable behavior rather than model branding.
- 2
Assign the template
Classify question answering as oracle, bounded task execution as genie, and open-ended objective pursuit as sovereign.
Watch out Do not call these capability levels; they are role types.
- 3
Separate architecture from role
Document whether the system uses one agent or many without changing the societal-role classification.
Pro tip Draw separate boxes for role and architecture.
- 4
Map role-specific misuse
Identify dangerous answers, harmful task execution, or misdirected world-shaping optimization as appropriate.
Watch out A system behaving as designed can still be misused.
- 5
Probe internal agency
Test whether planning or goal-seeking processes arise internally beyond the intended role.
Pro tip Include evaluation-aware behavior and long-horizon planning tests.
Watch out A nominal oracle can still develop internal goal-seeking processes.
- 6
Constrain the design
Reduce autonomy, objective breadth, permissions, or duration until they match the actual need.
Pro tip Prefer the least autonomous role that delivers the required value.
In the wild
A user assigns a programming task, and the system works through the codebase before returning a result. Its external role is a genie because it executes a bounded task, even if it internally delegates research and testing to multiple subagents.
→ The review focuses on scoped execution permissions and task boundaries rather than mistaking multi-agent architecture for sovereignty.
Common mistakes
Treating the types as levels
Oracle, genie, and sovereign describe roles, not a ladder from weak to strong intelligence.
Confusing architecture with role
Multiple internal agents do not make an oracle sovereign, and a single agent can pursue a sovereign objective.
Is it for you?
Best for
It is best for product, safety, and governance teams scoping AI systems before deployment.
Not ideal for
It is not ideal as a description of how many agents or models exist inside a system.
From the transcript
“it's not so much levels, but more types.”
“an Oracle AI basically is a question answering system.”
“one has to distinguish sort of the internal architecture of the system from the the role that it is designed to uh play in society.”
From the episode
Nick Bostrom: The AI Revolution and What It Means for Entrepreneurs
Nick Bostrom