Gating Turbulence Like a Rare-Event Cut: Consonants for Differentiable Vocal-Tract Synthesis

Physical speech synthesizers, which simulate air moving through a tube shaped like a vocal tract, reproduce vowels well and mangle consonants. Vowels are periodic: the vocal folds buzz at a pitch, and the tract shapes that buzz. Consonants such as /s/, /f/ and /t/ are bursts or hisses of turbulent air at a narrow point in the tract, and they have no pitch at all. Current fitting methods still ask for a pitch on every frame. So through an /s/ they drive the model with a buzz that should not be there, and they never learn to make hiss. We propose to give the synthesizer a second sound source, turbulent noise injected at the tract's narrowest point, and to switch it on with a gate designed the way rare-event physics experiments design background cuts. In those experiments, a few timing statistics of each detector pulse separate true events from look-alikes. The cut is then tuned to a fixed, calibrated rate of false alarms, because every spurious event is costly. For speech, the equivalent costly error is background noise mistaken for a consonant. That error makes the fitted tract invent constrictions and, in a prosthesis, produce spurious hiss.

The opportunity matters beyond audio quality. Brain-computer interfaces and voice prostheses that drive such synthesizers are currently limited to vowels. One intracortical system decodes only the two lowest vocal-tract resonances and explicitly lacks any way to control voicing onset or consonant release. If an analysis-by-synthesis fit (adjusting the synthesizer until its output matches a recording) can recover interpretable consonant controls, meaning when the turbulence source turns on, where the constriction sits, and how sharp the burst is, those controls become decoding targets that vowel-only systems do not have today.

Core Claim

Our claim is that vowel-only behavior in differentiable articulatory synthesis is mainly an excitation problem, and that a noise source switched by a leakage-controlled gate will restore consonant intelligibility where an always-on pitch-driven source cannot. We test a secondary, riskier claim as well. We expect gate features borrowed from pulse-shape discrimination to hold their false-alarm rate on noise types never seen in training better than a learned classifier does. Those features are timing statistics of each acoustic onset, combined with a harmonicity measure. We treat this second claim as an open question that the experiments must settle.

Grounding in Prior Work

The method builds on Articulatory Synthesis of Speech and Diverse Vocal Sounds via Optimization. That paper introduces VocalTrax, a differentiable JAX version of the Pink Trombone tube model. It fits tract area functions and glottal tenseness to recordings using pitch pre-extracted by the CREPE pitch tracker. VocalTrax reaches 48.7% automatic digit accuracy on AudioMNIST, against 11.0% for an unconstrained gradient inverter. The same paper acknowledges an audible gap and weak high-frequency turbulence. We keep its tube model and optimizer. What we change is the excitation: pitch is no longer imposed where there is no voicing, and a turbulent source becomes something the optimizer can fit.

Two vocoder papers motivate the claim that unvoiced excitation is the neglected part. Improving LPCNET-Based Text-to-Speech with Linear Prediction-Structured Mixture Density Network reaches a mean opinion score of 4.41 but restricts its distribution sharpening to voiced frames. Time-Domain Envelope Modulating the Noise Component of Excitation shapes noise only inside voiced frames. That paper also constrains our design. It found that a plain amplitude envelope over-damps noise and that the Hilbert envelope best matched natural phase statistics, so our turbulence source uses a Hilbert-style envelope. It also found no significant listener preference in naturalness tests, which warns us to measure consonant identification directly rather than rely on objective scores alone.

Two speech papers supply the discrimination ingredients. New harmonicity measures for pitch estimation and voice activity detection shows that harmonicity separates voiced speech (average 5.13) from environmental noise (1.10–1.75). By construction, it labels fricatives and stops as noise. Unvoiced speech segregation based on CASA and spectral subtraction segregates unvoiced speech by estimating the noise floor from neighboring voiced intervals. That estimate goes stale during long unvoiced stretches. Both papers show that periodicity alone cannot find consonants, and that a statistic describing the time profile of the unvoiced burst itself is missing.

The physics side supplies the gate design. First Results from the DEAP-3600 Dark Matter Search separates event types with $F_{\text{prompt}}$, the fraction of scintillation light arriving in a short early window, and reports electronic-recoil leakage below $1.2\times10^{-7}$. α-event characterization and rejection in point-contact HPGe detectors pairs an amplitude-to-energy cut with a slow-tail slope parameter. That pairing removes surface alphas while losing only 0.2% of true bulk events. We do not expect speech to reach physics-level leakage. What we borrow is the method: combine a few orthogonal timing statistics, and report how many true events pass at a fixed, calibrated false-alarm rate. No existing work connects these fields; the file found none.

Research Program

The system has three parts. The first is the synthesizer extension. We add a differentiable turbulence source to VocalTrax, injected at the location of minimum tract area. It has a learnable gain, a Hilbert-style amplitude envelope, and an onset time, and the glottal source is disabled wherever the gate is closed. The pitch-matching loss applies only on gated-voiced frames, which removes the dependence on CREPE through unvoiced stretches.

The second part is the gate. For each acoustic onset at time $t_0$ in a high-frequency band (above 2 kHz, following the CASA paper's boundary), we compute a prompt fraction:

$$F_p = \frac{E(t_0,\, t_0+\tau)}{E(t_0,\, t_0+T)},$$

Here $E(a,b)$ is band energy between times $a$ and $b$, $\tau \approx 10$ ms is the burst window, and $T \approx 80$ ms is the full segment. A stop burst gives high $F_p$. A sustained fricative gives an intermediate value. Stationary background noise has no sharp onset, so its value is near the uniform baseline $\tau/T$. We add a tail-slope statistic, analogous to the delayed-charge parameter, and the frame's harmonicity from the Yu–Wang measure, which plays the role of the orthogonal amplitude-to-energy cut. A two-dimensional cut on (harmonicity, $F_p$ and slope) assigns each segment to voiced, burst, frication, or background.

The third part is calibration. Following how physics experiments use calibration sources, we build background-only sets by injecting noise into TIMIT silences. We then set the gate so that at most 1% of background frames open the turbulence source, and report consonant acceptance at that operating point.

Baselines

The naive reference point is VocalTrax exactly as published: CREPE pitch on every frame and no fitted turbulence source. Its reported strongest number is 48.7% AudioMNIST automatic accuracy, with a human-identification odds ratio of 10.7 over Vocal-Tract-Grad, both from Articulatory Synthesis of Speech and Diverse Vocal Sounds via Optimization. On TIMIT, the paper reports only a qualitatively lower match error, so we found no reported TIMIT number on Lacuna and will re-run it ourselves. Beating this baseline means a statistically significant gain in AudioMNIST accuracy and in human consonant identification, under the same optimizer, iteration budget and loss weights.

The strongest competitor for the gate is a learned frame classifier. This is a small convolutional network trained on TIMIT manner-of-articulation labels, given the same training data and a matched tuning budget, and thresholded at the same 1% background-leakage point. A second competitor gates on CREPE's own voicing confidence, which costs nothing extra. An oracle gate built from TIMIT phone labels gives the upper bound.

Two ablations isolate the key ingredients. The first is a gate with no turbulence source, where the glottis simply switches off; it tests whether the gate alone helps. The second is a turbulence source that is always on with a freely learned gain and no gate; it tests whether gating matters at all. We claim success only if the gated-turbulence system approaches the oracle and clearly beats both ablations.

Why not just…?

Why not just use an end-to-end neural articulatory synthesizer? Highly Intelligible Speaker-Independent Articulatory Synthesis reaches about 10% word error rate on the Haskins corpus with WavLM features, a Conformer and a BigVGAN vocoder. For audio quality alone it would win. Its intermediate representation, however, is not a physical tube with an explicit noise source, and our goal is interpretable consonant controls usable by prostheses and brain interfaces. VocalTrax's own comparison shows that physical structure matters for inversion (48.7% vs 11.0%). We therefore compare within the physical-model family and report the neural system only as a quality ceiling.

Why not just gate on CREPE's confidence or a learned classifier? This is the real threat. In clean, in-domain speech a learned classifier may match or beat hand-built timing features. We state in advance that if it does, the contribution reduces to the gated excitation and the calibration protocol, and we will report it that way. The case for the physics-style features rests on held-out noise. We train on one set of noise types and test leakage on the unseen categories of the Yu–Wang set (airport, babble, subway and others) and on the IEEE-sentence noise types used by Hu and Wang. The test is whether the 1% leakage target holds. Harmonicity's large margin over noise gives some reason to expect robustness, but the experiment decides.

Why not just leave a learnable noise source on and let the optimizer sort it out? Gradient fitting is known to find degenerate explanations. With a physical tract, spurious noise can be absorbed by inventing constrictions. The always-on ablation tests this directly. We will measure whether it achieves similar digit accuracy and whether its fitted tract shapes during vowels stay open. If both hold, gating is unnecessary and we will say so.

Why not just denoise with spectral subtraction first? The CASA paper shows spectral subtraction alone trails its hybrid system by up to 9.3 dB at −5 dB input SNR. Its musical-noise residue falls in the same band as weak fricatives, which would be accepted as consonants. We include a spectral-subtraction front end in the noisy condition to measure this leakage.

Proposed Experiments

The decisive test uses AudioMNIST. Its digits are rich in stops and fricatives, as in "six", "seven", "three" and "five". We run clean and noise-mixed versions at 0 and 5 dB, measure automatic digit accuracy against the 48.7% baseline, and run a human identification study targeting consonant confusions. On TIMIT, phone labels give per-manner diagnostics: spectral distance and ASR phone error computed separately for stops, fricatives and vowels. We also check that vowel quality does not regress. For the gate, the headline figure is consonant acceptance at 1% background leakage, measured on in-domain and unseen noise types. The result is informative whichever way it falls. It shows either that timing-morphology cuts transfer, or that a learned gate suffices and the consonant gains come from the excitation model alone. In both cases the gate and constriction trajectories become candidate control targets for the vowel-only brain-interface decoder of Artificial speech synthesizer control by brain-computer interface. We leave that decoding step to follow-up work.