Shadow-and-Field Testing
Validate payment before building, then improve through direct real-world use
- Difficulty
- Moderate
- Time to result
- ~weeks to results
- Steps
- 6
- Confidence
- 98%
Shadow-and-Field Testing uses two complementary feedback loops. Shadow testing comes first: represent the proposed offer with a concept, page, prototype, or other credible description before building the final product. Present it to likely customers and ask the strongest question—whether they will pay—ideally through an order or conditional pre-order. If the demand signal justifies development, build enough to deliver the promise. Field testing then moves inside the operating product: the team uses what it makes in the intended context, notices failures with full situational knowledge, and acts without waiting for incomplete reports. Shadow testing validates willingness to buy; field testing accelerates learning about quality. Together they test the two major unknowns in sequence: whether customers want the offer and whether it works well in practice.
Origin
Josh Kaufman identifies shadow testing and field testing as his two primary ways to test the assumptions behind a new business.
Core principles
- 01Every business rests on assumptions that can be tested
- 02A payment commitment is stronger evidence than positive feedback
- 03Demand can be tested before the finished product exists
- 04Using your own product shortens the feedback cycle
- 05Fast contextual feedback improves quality
How to run it
- 1
Name the critical assumptions
List what must be true about price, demand, cost, and use for the business to work. Rank the assumptions by how fatal and uncertain they are.
Pro tip Test the assumption that can kill the idea before polishing secondary details.
Watch out A test without a stated assumption produces ambiguous learning.
- 2
Create a shadow
Build the smallest honest representation that lets a likely customer understand the promise. Use a concept page, prototype, mock-up, or demonstration.
Pro tip Represent the value and constraints, not a fictional level of readiness.
Watch out Never imply that an unfinished product already exists.
- 3
Ask for commitment
Show the shadow to likely buyers and ask whether they will order at a stated price. Record commitments, refusals, and the reasons behind them.
Pro tip A pre-order or signed order is stronger than a survey response.
Watch out Compliments and email signups do not prove willingness to pay.
- 4
Build the tested promise
Proceed only when the evidence meets the decision threshold. Build enough to fulfil what customers understood and purchased.
Pro tip Keep the first version aligned with the exact promise that earned commitment.
Watch out Do not add speculative scope between validation and delivery.
- 5
Use it yourself
Operate the product in a realistic workflow and note every failure, delay, and confusing interaction. Preserve the context around each observation.
Pro tip Use the product continuously enough to encounter edge conditions.
Watch out Founder use does not replace feedback from customers with different needs.
- 6
Shorten the feedback loop
Turn direct observations into small improvements, retest them, and continue collecting customer evidence. Prioritize failures that block the promised outcome.
Pro tip Fix the problem while its context is still fresh.
Watch out Fast iteration on the wrong promise does not create demand.
In the wild
A founder creates a one-page description and clickable mock-up for a scheduling assistant, states the monthly price, and asks ten consultants for conditional pre-orders. After three commit, the founder delivers the first version manually, uses the workflow for personal bookings, and records where confirmations and rescheduling break.
→ Payment evidence precedes development, while direct use quickly improves delivery quality.
Common mistakes
Testing opinions
Asking whether an idea sounds good is much weaker than asking customers to commit at a real price.
Misrepresenting readiness
A shadow test becomes deceptive when customers believe a nonexistent product is already available.
Treating founder use as universal
Direct use speeds learning but cannot represent every customer's environment or ability.
Is it for you?
Best for
It is best for founders who can represent an offer credibly before full development and later use it themselves.
Not ideal for
It is not ideal when pre-selling would mislead customers or when the maker cannot realistically represent the target user's context.
From the transcript
“there are two primary methods that i really like to use for for this”
“the strongest version of this test is you actually take orders from them”
“the best situations where the company improves to the greatest extent most quickly are very often the companies that use the thing that they themselves…”
From the episode
Josh Kaufman: Launching a Business or Side Hustle
Josh Kaufman