The Mr Potato Head Model of Sensory Substitution
Pipe any data stream into the brain through any channel and let it learn to perceive
- Difficulty
- Advanced
- Time to result
- ~months to results
- Steps
- 6
- Confidence
- 86%
Your eyes convert photons into electrical spikes; your ears convert air compression waves into spikes. The brain never receives light or sound, only spikes in the dark, and it constructs everything from there. Eagleman's Mr Potato Head model follows the logic: peripheral sensors are interchangeable plug-in devices, and the cortex is general-purpose enough that any of it can trade off with any other. So it does not matter how the information reaches the brain, as long as it reaches it with its structure intact and correlated with something real. Feed sound to the skin as patterns of vibration, and deaf users first decode, then simply hear. Feed infrared, drone telemetry or stock data through the same channel and a new sense grows. The build sequence is: choose a stream, choose a channel, encode structure, guarantee correlation, then expose the user daily while accuracy climbs from chance toward perception.
Origin
Extracted from Young and Profiting
Core principles
- 01The brain is locked in silence and darkness; every sense is already a construction from electrical spikes.
- 02Peripheral sensors are plug-and-play; the cortex is a one-trick pony that any data stream can drive.
- 03It does not matter how the information gets in, only that it gets in with structure.
- 04Correlation with the world is what turns a signal into a perception.
- 05Perception is earned over weeks of exposure, not delivered on day one.
How to run it
- 1
Choose the data stream
Identify information the person cannot currently sense but would act on if they could: ambient sound, infrared, a drone's pitch and yaw, a market feed.
Pro tip Pick a stream with real consequences attached. The brain learns what pays.
- 2
Choose an unusual input channel
Select a channel with spare bandwidth. Eagleman used the skin, first via a vest of vibratory motors, later a wristband. Earlier researchers used vibration on the forehead and pokes in the back via a modified dental chair.
Pro tip Skin is large, cheap to actuate and rarely saturated.
Watch out Do not hijack a channel the person already depends on for something critical.
- 3
Encode structure, not just presence
Map the stream so its internal structure survives. The wristband breaks sound from high to low frequency across the skin, which is exactly what the inner ear does.
Pro tip Copy the biological encoding where one already exists; evolution did the design work.
Watch out A signal that only says 'something happened' teaches nothing. The brain needs a pattern to correlate.
- 4
Guarantee correlation with the world
The signal must reliably co-occur with something the user can verify and act on, so the brain can conclude the pattern refers to something out there.
Watch out Noisy or intermittent correlation stalls learning indefinitely.
- 5
Expose daily and measure against chance
Test recognition from the first ten minutes and keep testing. Performance starts slightly above chance and rises roughly linearly with daily use.
Pro tip Publish the curve to users; visible progress sustains the weeks of practice the method requires.
- 6
Wait for the flip from decoding to perceiving
Around four months, users stop reasoning from the vibration and simply report the percept: not 'I felt something, maybe a dog' but 'I hear the dog'.
Watch out Judging the device at week two, while users are still consciously decoding, will make a working system look like a failure.
In the wild
Eagleman built a vest covered in vibratory motors that captured sound and broke it from high to low frequency across the skin, mirroring the inner ear. Deaf users learned to hear this way. He spun it out of his lab as Neosensory and shrank the vest to a wristband doing the same job. Wearers now identify their name being called, a doorbell, a baby crying, a dog barking. Testing showed recognition slightly above chance in the first ten minutes, improving linearly, until at roughly four months users described the experience not as feeling vibration but as hearing.
→ A theoretical neuroscience idea became a shipped product worn daily in deaf schools and by individuals worldwide.
Using roughly five dollars of infrared bolometers wired into the same wristband, Eagleman walks through a parking lot and feels the temperature of things around him. He can tell which cars have been parked a while and which arrived in the last twenty minutes from the engine block signature, and which of two chairs was more recently sat in. It is not deduction; it is information he simply has as he walks.
→ A sixth sense assembled from off-the-shelf parts, demonstrating the model generalises past substitution into addition.
Pitch, yaw, roll, heading and orientation of a drone are fed to the pilot's skin. Rather than reading instruments, the pilot feels the aircraft's state, describing it as having stretched their skin up to where the drone is. Pilots fly measurably better in fog and darkness. Eagleman notes he is implementing this with young engineers in Ukraine for defence use.
→ Performance gains in exactly the conditions where instrument-reading breaks down.
Common mistakes
Sending presence instead of pattern
A buzz that only signals an event gives the brain nothing to correlate. The stream's structure has to be preserved in the encoding or no perception forms.
Testing before the four-month flip
Early users are consciously decoding and will report the device as awkward. The linear improvement curve means the verdict at week two is not the verdict at month four.
Assuming the channel must resemble the original sense
Vibration on a forearm bears no resemblance to sound, and it works anyway. Designing for surface similarity rather than information structure wastes the model's core insight.
Is it for you?
Best for
Builders of assistive devices, human-augmentation hardware, or any interface trying to give someone intuitive access to data they currently have to read and interpret.
Not ideal for
Problems where the information is already perceived easily and the bottleneck is decision-making rather than input.
From the transcript
“it actually doesn't matter how you get the information into the brain as long as you get it there you can send information through a…”
“your brain is locked in silence and darkness and all it has are these billions of neurons sending electrical signals around”
“by about let's say four months people will describe it as hearing”
From the episode
David Eagleman: The Psychology of Time, How the Brain Shapes Reality & Human Nature
David Eagleman