A wearable silent-speech device reads electrical activity from the muscles of the face and neck while a person mouths words, and turns that activity into text. A recording of this kind is called electromyography (EMG). The device is usually too weak to run a large speech recognizer on its own, so it streams the signal over a radio link to a phone or server. That link is narrow, and on a battery-powered necklace every transmitted bit costs energy. This proposal argues that what should cross the link is not the muscle waveform but an estimate of the neural drive underneath it: how intensely the pools of motor neurons controlling each muscle are firing over time. A surface EMG trace is a sum of many overlapping, kilohertz-rate action potentials. In this note, "collisions" means these overlapping unit waveforms. Most of that fast detail is waveform shape, which carries little information about speech. The firing intensity that encodes articulation, by contrast, changes at the slow rates of speech movements. If an on-device encoder can see through the collisions to the drive, the stream that needs to be sent should shrink by orders of magnitude, and recognition accuracy should be preserved or even improve.
The same move addresses the field's worst problem, which is lack of data. Open paired EMG-and-speech data amounts to only about 20 hours across a few speakers, according to Confidence-Based Self-Training for EMG-to-Speech. The drive encoder, however, does not need speech at all to be trained. It can be pretrained on unlimited simulated superpositions of motor-unit waveforms whose true firing times are known, and then frozen or lightly tuned. In two sentences: compress wearable EMG on the device into a collision-resolved estimate of motor drive, trained from simulation rather than scarce paired speech, and transmit only that. We then measure word error rate as a function of transmitted bits per second against sending raw or feature-compressed EMG.
Core Claim
The claim is that, at an equal transmitted bitrate, a drive code learned from simulated motor-unit superposition gives lower word error rate on EMG-to-text benchmarks than two alternatives: noise-matched quantized raw EMG, and on-device handcrafted spectral or time-domain features. The advantage should grow as the bit budget shrinks and should be largest for silent speech, where macro-level features perform worst. The current reference point is the 48.29% session-dependent word error rate for silent speech on the EMG-UKA corpus, against 26.90% for audible speech in the same evaluation (The EMG-UKA corpus).
The analogy from radio networks is precise enough to borrow design rules from. In centralized radio access networks, remote antennas either forward raw digitized samples or do some processing locally. The raw transport rate grows as channels × sampling rate × bits per sample (Cloud Radio Access Network). Sending raw EMG to a server is the equivalent of forwarding raw samples. Sending the drive estimate is closer to decoding locally and forwarding the extracted content.
Relation to Prior Work
The closest work falls into three groups, each missing one piece. Biosignal speech papers attack data scarcity but treat EMG as a bulk spectrogram. CoM2S generates synthetic EMG from soft speech units with a GAN and filters it by teacher confidence, reaching 21.87% word error rate on the Gaddy & Klein benchmark (Confidence-Based Self-Training for EMG-to-Speech). DiffMV-ETS synthesizes speech in selectable voices without target-speaker audio. Towards EMG-to-Speech with a Necklace Form Factor shows that dry neck electrodes streaming over Bluetooth can classify an 11-word vocabulary at 92.7%. None of these models motor-unit superposition, and none studies what the radio link should carry. The necklace paper motivates our setting, and CoM2S supplies a baseline and a synthetic-data component. CoM2S also constrains our design, because it reports a persistent synthetic-to-real gap (8.75% vs. 21.87% WER). We therefore use simulation only to train a speech-agnostic drive encoder, not to stand in for real speech data.
Collision-resolution work supplies the method, but it assumes unlimited bandwidth and heavy compute. Several lines of work recover individual firing from overlapping waveforms:
- Iterative subtraction: Intramuscular EMG Decomposition repeatedly removes the dominant waveform and re-examines what is left.
- Blind sparse deconvolution: A blind sparse deconvolution method jointly estimates the unit waveforms and their firing times.
- Real-time state-space decoding: Real-Time Decomposition of Multi-Channel Intramuscular EMG achieves over 90% accuracy, but needs server GPUs and receives all channels losslessly.
Our encoder does not perform hard decomposition. Hard decomposition from a few dry surface electrodes on facial muscles is unlikely to succeed, and it is too expensive for a necklace. Instead, we borrow the training signal that CEED used: augmentations that simulate collisions, jitter and noise, which pushed its encoder to an Adjusted Rand Index of 0.78–0.89 against 0.28–0.49 for denoised PCA. Our difference from CEED is the target. CEED learns invariance to collisions for spike sorting. We regress the latent drive itself and compress it for a downstream speech task. Simulation tooling in the style of SimSort motivates generating the training data at scale.
Radio-network theory supplies two design rules. First, Uplink Multi-Cell Processing shows that under a shared backhaul budget, quantization noise should be scaled to the background noise level. We adapt this by setting the quantization step of the drive code to match the encoder's own estimation error, measured in simulation. Second, Control-Data Separation across Edge and Cloud shows that making control decisions locally while forwarding data for central decoding is robust to link delay. Our analogue is that the device decides when the user is articulating and how many bits to spend, while the server does all recognition. These papers have never been applied to biosignals. We use them as design rules, not as theorems that transfer directly.
Research Program
Simulator. We first build a superposition simulator for surface EMG of the speech articulators, following the generative model $y(t)=\sum_i \psi_i(t-\tau_i)+\eta(t)$. Here $\psi_i$ is the waveform unit $i$ produces at an electrode, $\tau_i$ are its firing times, and $\eta$ is sensor noise. Firing times are drawn from rate processes driven by smooth articulator-like control signals. Unit waveforms are randomized over shape, amplitude, electrode-skin conditions and slow drift, so that the encoder must generalize across sessions. The simulation target is the drive $d_m(t)$: the smoothed summed firing rate of muscle pool $m$.
Encoder. We train a small causal encoder, sized to run on a microcontroller-class or phone-class chip, to map multichannel EMG to $\hat d(t)$, a low-dimensional drive code at a few tens to a hundred frames per second. The code is quantized with a step tied to the encoder's validation error. We also train a variant with a learned entropy model, so that the bitrate is an explicit knob. On the device, a cheap detector of articulation onset gates transmission; this is the "control" half of the split.
Server-side recognizer. The server runs an EMG-to-text model trained on real paired data plus, optionally, CoM2S-style synthetic data. It takes the decoded drive code as input instead of raw EMG.
Encoder validation. Before any speech experiment, we check the encoder against ground truth on the simulated and human intramuscular datasets of Real-Time Decomposition of Multi-Channel Intramuscular EMG, where true spike trains or decomposed spike trains exist. The encoder sees only a degraded, surface-like subset of channels and must recover pool firing rates. This tests whether "drive" is a real recoverable quantity or an artifact of our simulator.
Baselines
Raw-rate reference. The naive reference is the same server-side recognizer fed raw EMG at full resolution, with no constraint on the link. This sets the accuracy ceiling for an unconstrained link. On Gaddy & Klein, the strongest reported number in the file is 21.87% WER for CoM2S (Confidence-Based Self-Training for EMG-to-Speech). We will reproduce this recipe, including the 1:1 real-to-synthetic mix, as our full-rate recognizer, so that every compressed condition is compared against the strongest existing system rather than a weak one.
Noise-matched raw quantization. This is the strongest communications-style competitor. Raw EMG is downsampled and uniformly quantized with the step set at the sensor noise floor, following the rule in Uplink Multi-Cell Processing. It is then entropy coded with the same learned entropy model the drive code uses. The recognizer is retrained at each bitrate.
On-device features. This is the strongest signal-processing competitor. It computes handcrafted time-domain features of the kind behind the EMG-UKA HMM results (TD10 features with linear discriminant analysis), or EMG spectrograms like the necklace work, on the device. These are quantized to the same bit budgets.
Ablations. We isolate the key ingredient with two ablations. The first trains an encoder of identical architecture and bitrate with a reconstruction objective on real EMG, with no simulator and no drive target. The second uses CEED-style contrastive collision augmentation without the drive regression target.
Fairness. All conditions use the same real training data and the same recognizer architecture, and have the same hyperparameter search budget at each rate point. Encoders are restricted to the same on-device compute budget.
What counts as winning. Beating a baseline means a lower WER at matched bits per second across the low-rate half of the curve. Equivalently, it means matching full-rate WER (within the reproduced CoM2S number's run-to-run variance) at a rate the baselines cannot reach without degrading.
Why not just…?
Why not just send noise-matched raw EMG? For a few channels at kilohertz sampling, this may already fit a Bluetooth link. It is the right answer if bitrate does not actually bind, and we take that threat seriously. Our response is to report the full rate–WER curve and the radio energy implied by each point, rather than a single operating point. If noise-matched raw quantization matches the drive code at every rate down to where the radio's energy cost becomes negligible, the claim fails. We would then publish that result, because it would tell hardware designers they need not compress cleverly. The case for the drive code rests on the slow timescale of articulation. Raw quantization pays for kilohertz waveform detail no matter how good the quantizer is.
Why not just compute standard EMG features on the device? Time-domain features are already low-rate. They are the strongest practical threat, because they are a crude form of "send the drive." The evidence that they lose something is indirect: the silent-speech cliff on EMG-UKA (48.29% vs. 26.90% audible) suggests macro features miss information when acoustic feedback is absent. That is suggestive, not proof. The matched-bitrate comparison on the EMG-UKA silent subset settles it. If features tie the drive code there, the simulator adds nothing, and the honest conclusion is that hand-designed features were already near the right representation.
Why not just run the whole recognizer on the device? The hardware asymmetry is severe. Real-time motor-unit decomposition alone needed server GPUs in Real-Time Decomposition of Multi-Channel Intramuscular EMG, and large-vocabulary transducers are far heavier than a necklace module can run. Splitting at the drive code keeps the device cheap and puts the language-dependent, data-hungry part where it can be updated, for example with LLM rescoring as in In-context Language Learning for Endangered Languages.
Why not just add more synthetic EMG, as CoM2S does? That is complementary, not competing. CoM2S synthesizes EMG conditioned on speech and inherits a synthetic-to-real gap in the speech mapping. Our simulator never claims to model speech; it only teaches signal-level unmixing. The factorial experiment below, with and without Libri-EMG data and with and without the drive encoder, shows whether the gains add up.
Proposed Experiments
Rate–WER curves. On Gaddy & Klein, with the CoM2S reproduction as the full-rate anchor, every condition in the Baselines section is evaluated across a sweep of bit budgets, down to roughly one bit per drive dimension per frame. On EMG-UKA, we evaluate audible, whispered and silent modes separately. We also evaluate in the session-independent setting, whose reported audible reference is about 27.02%, because simulated waveform variability should help cross-session robustness. The prediction is that the drive code's margin over the baselines is largest in silent and session-independent conditions and at low rates.
Encoder fidelity test. On the intramuscular datasets, we measure correlation between estimated and true pool firing rates as electrodes are removed and noise is added. If the encoder's speech gains appear without fidelity to true drive, the "drive" interpretation is wrong, even if the system works.
Factorial data experiment. This crosses the drive encoder (on or off) with Libri-EMG augmentation (on or off) at a fixed low bitrate.
Delay robustness test. Following the control-data separation analysis, we add link delay and packet loss. We compare local gating against server-side gating, a direct biosignal analogue of the edge-versus-cloud control comparison.
Kill condition. The proposal fails cleanly if, at equal bitrate, the reconstruction-trained ablation matches the drive encoder. That outcome would mean the collision simulator contributes nothing beyond generic learned compression.
References
- Confidence-Based Self-Training for EMG-to-Speech: Leveraging Synthetic EMG for Robust Modeling
- The EMG-UKA corpus for electromyographic speech processing
- Cloud radio access network: Virtualizing wireless access for dense heterogeneous systems
- DiffMV-ETS: Diffusion-based Multi-Voice Electromyography-to-Speech Conversion using Speaker-Independent Speech Training Targets
- Towards EMG-to-Speech with a Necklace Form Factor
- Intramuscular EMG Decomposition Basing on Motor Unit Action Potentials Detection and Superposition Resolution
- A blind sparse deconvolution method for neural spike identification
- Real-Time Decomposition of Multi-Channel Intramuscular EMG Signals Recorded by Micro-Electrode Arrays in Humans
- Towards robust and generalizable representations of extracellular data using contrastive learning
- SimSort: A Data-Driven Framework for Spike Sorting by Large-Scale Electrophysiology Simulation
- Uplink multi-cell processing: Approximate sum capacity under a sum backhaul constraint
- Control-Data Separation across Edge and Cloud for Uplink Communications in C-RAN
- In-context Language Learning for Endangered Languages in Speech Recognition