Part IV of VII6 min read
The interface is a learning problem
Hardware is the foundation, data is the fuel, and the system that wins is the one that learns each person fastest. Only EEG can do all three at the scale of a population.
Implants give the richest signals and every headline result for speech. Each one is also a neurosurgical operation. Their reach is bounded by the number of surgeons and by how much risk a person’s situation justifies. For severe paralysis that is clear. For everyone else it is not, and it will not be for a long time.
A device without surgery can be redesigned in weeks, tested on thousands of people and shipped without a hospital. So the real question is narrow, and experiments can answer it: how much useful information can be captured from outside the skull, and how cheaply can a model be adapted to a new person on a new day?
The unoccupied corner
Plot every sensing method by how wearable it is and how rich a signal it yields for deliberate communication. One corner of the map is empty. We think EEG, trained at scale, is what moves into it.
- Brain, no surgery
- Open surgery
- Catheter-delivered
- Peripheral, not brain
- EEG. Wearable; richness limited by the skull. Robust for a few commands, weak for open text so far.
- fNIRS. Wearable; slow blood signal carries little per second.
- MEG. Best non-invasive language results, inside a shielded room.
- fMRI. Rich in space, slow in time, immobile.
- ECoG. Goes everywhere once implanted; getting there is brain surgery.
- Intracortical. Richest signal recorded; surgical, small coverage, durability open.
- Endovascular. Less surgery, fewer channels, small command sets to date.
- sEMG. Rich for deliberate movement only; needs working muscle. Not a brain signal.
- Radar / motion. Contactless; early evidence; senses movement, not intent.
The ceiling is set at the sensor
Picture the field as four layers. Hardware at the bottom: what the sensor registers. Then data: what has been recorded. Then algorithms. Then products. Attention and money pool at the top, where the demonstrations live. The constraint runs the other way.
Information theory states the reason bluntly. No computation on a signal can add information about its source. A decoder can waste what the sensor captured. It cannot create what the sensor missed.
Hardware is the actual bottleneck, and product is what people are focusing on. That mismatch is the state of the field.
Four ceilings
Each layer limits the one above it. Information about what a person intended can be lost on the way up and never gained. Select a layer.
↑ Information about intent can only shrink
+ Fluency from a language model. Looks like more. Is not more signal.
Hardware sets what exists
- Bounds
- The information present in the recording. Everything above works with this and nothing else.
- Cannot be fixed by
- More data or a larger model.
- Evidence
- On one task across four devices, the more precise sensor decoded better at matched data.34
Data sets what has been seen
- Bounds
- How much of what the sensor can capture has actually been observed, across people, days and tasks.
- Cannot be fixed by
- Cleverer models, past a point.
Algorithms set what is extracted
- Bounds
- How much of the recorded information is recovered, and how fast a model adapts to a new person.
- Cannot
- Add information the recording lacks.
Products set what arrives
- Bounds
- How much of the extracted information reaches a person as something useful: correction, comfort, trust.
- Cannot
- Rescue a missing signal. Can waste everything beneath it.
- Evidence
- The four gates of Part III.
Why EEG wins
One experiment ran the same task, working out which image a person is looking at, on four kinds of machine: EEG, MEG and two strengths of fMRI. Eight datasets, 84 people, 498 hours.34 Two results matter. Deep learning helped most on the noisiest sensor, which is EEG. And no sensor showed a plateau as data grew.
MEG has a cleaner view and needs a magnetically shielded room.30 A technology that cannot leave a special facility cannot learn from a million people. fNIRS is wearable, but it measures a blood response that arrives seconds late, too slow for anything at the speed of speech.
EEG is fast enough, cheap enough and safe enough to be worn by anyone, anywhere. Its weakness is noise, and noise is exactly what modern learning is best at overcoming. At equal data the precise instruments lead. But the data will never be equal. A sensor that costs a thousand times less and goes home with the user gathers thousands of times more, and the history of machine learning is the history of the scalable method beating the elegant one.
Building EEG for scale changes the design. Portability comes first. Other sensors that do not interfere are stacked beside it. Sensors and models are designed as a pair, so the model’s results decide what the next sensor measures. Cameras and microphones sit on the device, because context is what gives a recording its meaning.
A field without its ImageNet
Modern AI was built on collections nobody had to commission. The image dataset that launched deep learning gathered millions of photographs already on the web.59 A speech recogniser released in 2022 learned from 680,000 hours of audio collected from the internet.42
Brain data cannot be scraped. Every hour needs a consenting person under a sensor. So this field counts in hours where its neighbours count in lifetimes.
Hours, not lifetimes
The largest neural datasets behind published decoders are hundreds to thousands of hours. A single speech model was trained on 680,000.
Public datasets exist and matter.295758 They differ in sensor, layout, task and file format. Pooled, they are a patchwork. The field grew up inside medicine, where the habit is a small careful study and an algorithm built by hand for one patient.
There is also a trap. Hardware makers will not scale until there is demand for data. Data collectors cannot gather volume until hardware is cheap. Model builders cannot justify large training runs until the data exists. Each layer is sensibly waiting for the one beneath it.
The way out is to build every layer at once, ahead of demand. That is the case for a full-stack hardware company, and it is the company Setu is building. The one biosignal system that already works on strangers came from exactly this kind of integrated effort.19 It does not have to be a solitary one: shared formats and shared benchmarks make every stack better.
Scale is the strategy
AI learned this the hard way, and one of its founders named it the bitter lesson: general methods that scale with data and computation end up beating methods built on human cleverness.61 BCI does not need seventy years to learn the same thing.
The measurements agree. Decoding improves steadily as recordings grow, for images34 and for sentences.16 Larger EEG models do not yet beat small ones, which tells us the models are starved, not that they are wrong.262728 And more hours from one person help more than more people do,34 which is an argument for a device worn every day.
None of this requires understanding the brain first. Nobody fully knows how it computes. The wager is that enough structure can be pulled from its signals without a complete theory, as it was for language. Nor does it require new mathematics. The methods were worked out once, for language models, and they carry over.
The playbook, transplanted
Ten techniques that turned pretrained AI models into useful ones, each with its counterpart in biosignal decoding, the state of the evidence, and what goes wrong.
| In AI | Counterpart in biosignal decoding | The catch |
|---|---|---|
| Self-supervised pretraining | Train on unlabelled recordings pooled across people and sessions: masked-signal models for EEG, multi-session models of spiking.4140 | In benchmarks, larger models were not reliably better and specialists stayed competitive.27 |
| Supervised fine-tuning | Train on a labelled task with known ground truth: cued speech, typed sentences, viewed images. | Labels need a ground truth. For private mental states there is none to check against. |
| Transfer across people | A shared model plus a light alignment per person;13 a generic wrist-signal model.19 | For image decoding, adding people helped little; adding hours per person helped.34 |
| Few-shot personalisation | Minutes to an hour of a new person’s data.913 | The burden lands on the user, and returns if drift is unmanaged. |
| Continual learning | The decoder keeps updating during ordinary use; one ran 403 days without a retraining session.359 | Self-training on corrected guesses can drift toward the language model. |
| Test-time adaptation | Re-align each day’s activity to a stable underlying structure, with no labels.3637 | Assumes the structure is stable. Unproven for signals recorded through the skull. |
| Language prior and search at inference | A language model rescoring the decoder’s candidates.89 | Adds fluency, not information. Report results with and without it. |
| Learning from human feedback | Corrections and confirmations as labels; error-related brain responses as a reward signal.49 | Implicit reactions are noisy and are not consent. See Part V. |
| Co-adaptation | No counterpart in language models: the person’s brain adapts to the decoder while it adapts to them.3839 | Both sides are moving targets. Stability has to be designed. |
| Distillation, on-device inference | A small personal model that runs locally. | Wanted for privacy and delay; little published evidence yet for brain decoders. |
One row deserves a closer look. An implanted handwriting decoder stayed accurate for 403 days without stopping its user to recalibrate. A language model corrected its output, and the corrections were fed back as training labels.35 Others stay stable with no labels at all, by re-aligning each day’s activity to the structure that persists beneath it.3637
There is a trap in it. A decoder trained on its own corrected guesses learns from the language model as much as from the person. It can grow more fluent and less faithful at once. The cure is to anchor learning to what the person explicitly confirmed.
Deliberate confirmation is not only a safeguard. It is the cleanest training signal such a system will ever receive.
How long before it knows you
One number says more about a system’s future than its headline accuracy: how much of a person’s time it needed before it worked.
How long before it knows you
The personal data each published system needed before it worked for a given person. The spread runs from nothing to dozens of hours.
This is why continuous adaptation is the way to build, and why training a model once and launching it is not enough. People differ. One person’s signals change from day to day. So the architecture has two parts: a large general model that knows what brain signals are like, and a small personal layer, fitted in minutes and updated continuously, that knows what yours are like.
If every user is fitted individually, nobody is the default user. Not one language, one head shape or one way of moving.
The stranger problem
Train a decoder on one person on one day, then ask it to work on someone else, or on the same person next week. Choose a training cell and switch three remedies on and off.
Trained on person C, day 3. It works there, fades across C’s other days as sensors shift and the brain drifts, and fails on everyone else.
Anchors in the literature: a per-person fMRI language decoder did not transfer to other people;11 shared pretraining cut new-person fMRI data to about an hour;13 an implant recalibrated in minutes;9 a wrist EMG model generalised with no calibration;19 an implanted handwriting decoder recalibrated itself for 403 days.35 Each is a different signal and task.
Two learners
In one respect this is unlike the rest of machine learning. A language model’s training data never learned anything back. Here the source of the data is a brain, and brains adapt. In animals, neural activity reorganised around a decoder as it was used.38 In people, months of mutual training took two pilots with tetraplegia to an international competition for brain-controlled devices, with learning measured on both sides.39
The right picture is a musician and an instrument coming to fit each other. The system must be stable enough for the person to learn it and adaptive enough to follow the person as they change.
Two loops
A device that is useful gets worn. Wearing produces data and, more valuable, outcomes: this decode was accepted, that one was corrected. Those improve the model, which shortens calibration, which makes the device useful to more people.
A second loop turns around the first. A product people keep wearing earns revenue. Revenue and results attract capital and talent. Capital and talent buy better sensors and more research. AI followed exactly this path. No field has scaled on grants alone.
Two loops, one device
A device people choose to wear turns two wheels at once. The inner one makes the technology better. The outer one pays for it.
Inner loop · the technology
- A useful applicationSomething narrow that people choose to use every day.
- Consented data with outcomesMany people, many days, with every confirmation and correction recorded.
- Better modelsGeneral representations that tolerate new heads and new days.
- Less calibrationMinutes instead of hours to learn a new person.
- Wider useMore people for whom it is worth wearing, and more applications.
Outer loop · the means
- A product people keep wearingRetention is the first proof that the value is real.
- RevenueSales that do not depend on the next grant or the next round.
- Capital and talentResults and revenue draw investors, engineers and scientists into the field.
- Better sensors and researchManufacturing at volume, lower cost per device, and funded science, feeding the next product.
Download the full manifesto as a PDF
Setu Glasses · Setu Cap