Objective and Trade-off Specification
Define the outcome and price every error before optimizing the system
- Difficulty
- Advanced
- Time to result
- ~weeks to results
- Steps
- 5
- Confidence
- 97%
Modern AI can optimize a stated objective, but it cannot decide which objective society or a business ought to value. Start by defining the desired outcome, then identify the system's unavoidable false positives and false negatives. Make the relative cost of those mistakes explicit: who is harmed, how severely, and what threshold is acceptable? This turns a hidden value judgment into a reviewable design decision. The mechanism is objective clarity followed by error mapping, stakeholder valuation, and threshold selection. It is especially important in high-stakes systems, because an apparently technical setting can encode a consequential moral or commercial choice. The framework shifts attention from improving algorithms in isolation to specifying what success means before optimization begins.
Origin
Peter Norvig described how AI progress has shifted from algorithms and data toward the harder task of deciding what objective a system should optimize.
Core principles
- 01Optimization is only useful after the objective is clear
- 02Every imperfect system forces trade-offs between different errors
- 03Value choices cannot be delegated to an algorithm
- 04Exact thresholds deserve explicit human scrutiny
How to run it
- 1
Name the real outcome
Write the human or business result the system should produce, rather than the convenient metric it can easily maximize.
Pro tip Use a plain-language sentence that a nontechnical stakeholder can challenge.
Watch out A precise metric can still represent the wrong goal.
- 2
Map both error types
Describe what a false positive and a false negative look like in the real world.
Pro tip Use concrete cases rather than statistical labels alone.
Watch out Do not assume one error is harmless because it is less visible.
- 3
Identify who bears the cost
List the users and stakeholders affected by each error and the consequence for each group.
Pro tip Include people who never operate the system but live with its decisions.
Watch out Optimizing for the direct user can externalize harm onto everyone else.
- 4
Choose the trade-off
Set an explicit threshold that reflects how the competing mistakes are valued.
Pro tip Record the rationale so the number can be reviewed later.
Watch out The system will require an exact boundary even when the underlying value judgment is uncomfortable.
- 5
Test the objective in practice
Observe whether optimizing the chosen objective creates the intended outcome across affected groups, then revise the threshold when evidence changes.
Pro tip Review outcomes by stakeholder group, not only in aggregate.
Watch out Reliable software can still reliably optimize the wrong objective.
In the wild
A court uses an AI aid to estimate whether a defendant will reoffend. The team cannot eliminate mistakes, so it must decide how to weigh detaining someone who would not reoffend against releasing someone who does. Rather than hiding that choice inside a risk score, it defines both errors, identifies the defendant, victims, families, and society as stakeholders, and makes the operating threshold explicit.
→ The system's threshold becomes a visible policy choice rather than an invisible technical default.
Common mistakes
Optimizing before defining success
A capable algorithm will efficiently pursue whatever objective it receives, including a poor proxy for the real outcome.
Treating the threshold as neutral
An exact decision boundary embeds a judgment about whose errors matter and by how much.
Is it for you?
Best for
Teams building AI or decision systems where mistakes affect people differently.
Not ideal for
Simple deterministic tasks whose correct output and failure costs are already unambiguous.
From the transcript
“really the key to Future progress is neither of those the key is deciding what is it that you want what is it that you're…”
“you got to tell me what the objective is what is it that you're trying to do”
“what's the tradeoff between those mistakes”
From the episode
Peter Norvig: Simple Ways to Grow Your Business with AI
Peter Norvig