Self-Similarity Tests for Neural Glottal Inverse Filters

When we speak, the sound that leaves the lips is the airflow pulses from the vocal folds (the "source") passed through the resonant tube of the throat and mouth (the "filter"). Recovering the source from the recorded sound is called glottal inverse filtering. It is an ill-posed unmixing problem, and it has almost no ground truth: nobody can measure airflow at the vocal folds of a talking person without invasive equipment. The most promising recent approach trains a neural network on pairs of synthetic speech and airflow produced by a physics simulator of the vocal tract, then applies the network to real speech. The catch is that nobody can say where on real speech such a network should be trusted, or whether it has learned quirks of the simulator rather than physics.

We borrow the answer cosmologists gave to an analogous problem. When a simulated universe has no built-in length scale, snapshots taken at different times must collapse onto one curve after rescaling, and any failure to collapse measures numerical error without needing the true answer. Linear source-filter acoustics has an exact symmetry of the same kind. If a recording is played back slowed by a factor $\lambda$, it is exactly the sound of a speaker $\lambda$ times larger, and the true source of that slowed signal is the original source slowed by $\lambda$. A correct inverse filter must therefore commute with time-rescaling. We propose to measure how far trained inverse filters break this rule, on real and synthetic speech, and to put resampling-based confidence bands on the breakdown. The result is a label-free map of where in pitch and frequency an estimate can be believed. We then test that the map predicts true error on the few benchmarks where ground truth exists.

Core Claim

Time-rescaling a convolution rescales both factors. If $s = e * v$, where $s$ is the recorded pressure, $e$ the glottal source, and $v$ the tract-plus-lip impulse response, then $s(t/\lambda) = e_\lambda * v_\lambda$, with $e_\lambda(t) = e(t/\lambda)$ and $v_\lambda(t) = \lambda^{-1} v(t/\lambda)$. Any separator $G$ that maps speech to source should therefore satisfy $G[s(\cdot/\lambda)](t) = G[s](t/\lambda)$, up to a known amplitude factor when $G$ outputs the flow derivative. We define a rescaling deviation as a function of frequency measured in units of the fundamental frequency $F_0$, $\tilde f = f/F_0$:

$$D_\lambda(\tilde f) = \frac{\left|\widehat{G[s_\lambda]}(\tilde f) - \widehat{G[s]_\lambda}(\tilde f)\right|}{\left|\widehat{G[s]_\lambda}(\tilde f)\right|}.$$

Here hats denote short-time spectra, $s_\lambda$ is the rescaled input, and comparisons are restricted to the band both versions share below the lower Nyquist frequency. This plays the role of $\Delta\xi/\xi$ in Quantifying resolution in cosmological N-body simulations using self-similarity, which measured deviations of rescaled two-point correlation functions across expansion times to locate sub-percent convergence boundaries without a reference solution. In that paper, rescaled separation $r/R_{\mathrm{NL}}$ is the dimensionless axis. Here the axis is $\tilde f$ together with the ratio of pitch to first formant, $F_0/F_1$.

The claim has two parts. First, $D_\lambda$ computed on unlabeled speech predicts the true source-estimation error of a network trained on simulator data. Second, the region where its bootstrap lower band stays under a tolerance is a usable trust region whose boundary moves in interpretable ways. We expect the boundary to move with the $F_0 > F_1$ regime, where Quasi closed phase analysis for glottal inverse filtering documents classical breakdown, and with the range of tract lengths and sampling rates seen in training.

What is new is not equivariance as a training trick. Vocal-tract-length perturbation is old. The new part is using the symmetry as a measurement instrument, with statistical confidence, for an inverse problem that lacks ground truth.

Research Program

The first component is the probe applied to trained separators. We will reproduce the BiLSTM of Glottal inverse filtering based on articulatory synthesis and deep learning. That network was trained on roughly 120 minutes of VocalTractLab speech/airflow pairs at 8 kHz, generalized qualitatively to German speech, and beat IAIF (iterative adaptive inverse filtering) on OPENGLOT. Its authors treated the simulator output as exact and gave no uncertainty for the predictions. We will compute $D_\lambda$ for $\lambda$ between about 0.7 and 1.4, the range spanning child-to-large-adult tract lengths, on four kinds of input:

  • held-out VocalTractLab speech;
  • OPENGLOT;
  • the 63,750-vowel physical-modeling database used by Airaksinen, Story and Alku;
  • real speech from BITS, plus high-pitched singing from GTSinger.

Rescaling is done by reinterpreting the sample rate and resampling back to 8 kHz. Comparisons are band-limited, which mirrors the role of the grid Nyquist frequency $k_\Lambda$ in Accuracy of power spectra in dissipationless cosmological simulations. The same probe applies unchanged to QCP, to IAIF, and to the GCI-free NMF method of Automatic Glottal Inverse Filtering with Non-Negative Matrix Factorization.

The second component is the confidence band. The self-similarity literature declares breakdown at fixed percentage thresholds and never asks whether a deviation exceeds sampling noise. In speech, the noise comes from which utterances, glottal cycles and turbulence realizations happened to be analyzed, and the realizations are correlated in time. We will use a two-level bootstrap. Utterances are resampled first. Within each utterance, contiguous blocks of glottal cycles are resampled, since independent frame resampling would yield the artificially narrow bands that the resampling literature warns about. Simultaneous bands over $\tilde f$ and $F_0/F_1$ come from a multiplier bootstrap. Here we reuse the construction that Confidence Bands for Multiparameter Persistence Landscapes justified via a functional central limit theorem for Lipschitz-stable summaries, and that Gaussian Approximation and Multiplier Bootstrap for Stochastic Gradient Descent analyzed for iterative estimators. Our spectral deviation is not proven Lipschitz-stable in their sense. We will therefore check coverage empirically on synthetic data, where the true deviation can be computed from many independent simulator draws. The trust region is the set of $(\tilde f, F_0/F_1)$ where the upper band of $D_\lambda$ stays below a tolerance $\tau$. Its boundary carries a bootstrap interval.

The third component is the same symmetry applied one level up, to the simulator that generates the training data. This is the gap the research file identifies: articulatory simulators have never been convergence-tested, yet their output is used as ground truth. The lossless linear tube equations are scale-free under joint rescaling of tract length and time. VocalTractLab's frequency-dependent wall, viscous and radiation losses, and its fixed sample rate and section count, break that symmetry partly physically and partly numerically. We will drive the tract with a prescribed source, either through the synthesizer or through a reimplementation of its tube model if a prescribed source is not exposed. We then compare runs that are rescaled with losses scaled to keep their dimensionless form against runs at refined section count and sample rate. This separates numerical breakdown from physical scale-breaking.

The fourth component closes the loop. If the simulator probe finds an unresolved band, we retrain the separator after filtering out or reweighting that band. We then ask whether the network's own $D_\lambda$ and its true error both improve.

As a calibration of the bootstrap machinery on a problem with a known answer, we will apply the same spatial block bootstrap to rescaled two-point correlations from the ABACUS scale-free suite. The check is whether the bootstrapped boundary reproduces the reported sub-percent region and its $a^{-1/2}$ late-time contraction. This is a sanity check and not a cosmology contribution.

Baselines

The naive reference is a fixed trust rule: trust everything below 4 kHz and below $F_0 = 200$ Hz, the threshold above which QCP was reported to matter most. If $D_\lambda$ cannot beat this rule at flagging high-error frames, the probe adds nothing.

The strongest competing separator is the simulator-trained BiLSTM of Langheinrich et al.. Among classical methods, the strongest is QCP, which achieved the lowest errors on NAQ, QOQ, HRF and H1–H2 in its source paper. IAIF serves as the common reference. There is no reported result found on Lacuna for the BiLSTM's correlation or open-quotient error on OPENGLOT, nor for QCP's absolute error values. We will therefore rerun all separators ourselves on identical splits at 8 kHz, with matched tuning effort for QCP's weighting parameters and for the network. The scored quantity is ground-truth source error on OPENGLOT and the Airaksinen–Story–Alku database.

The strongest competitor for the uncertainty question itself is a deep ensemble of five BiLSTMs, with disagreement used as the error predictor. It is trained on the same data and compute as five single models. The probe beats it if $D_\lambda$ achieves higher rank correlation with true per-frame error and higher area under the curve for flagging frames whose NAQ error exceeds a fixed threshold. This must hold out of distribution, on the second physical simulator and on high-$F_0$ vowels. We also test combinations of the two predictors. If the ensemble wins everywhere, the central claim fails.

Two ablations isolate the key ingredients. The first replaces block resampling with independent frame resampling and measures the loss of coverage. The second compares a network trained with resampling augmentation, so that equivariance is partly enforced, with an unaugmented one. The test is whether residual $D_\lambda$ still predicts error after augmentation.

Why not just…?

Why not just refine the simulator and check convergence, as engineers always do? For a one-dimensional tube model, refinement is cheap, so the cost argument that motivates self-similarity in cosmology does not apply here. We will run refinement, and if it finds every artifact the rescaling test finds, the simulator-side probe is redundant and we will say so. Two things survive regardless. First, refinement can show only that the model has converged to itself. As Quantifying resolution in cosmological N-body simulations using self-similarity showed, successive refinements can agree while sharing a common bias, and rescaling runs with losses held in dimensionless form give an independent check. Second, and decisively, refinement says nothing about the trained network on real recordings. The separator-side probe is the core of the proposal for that reason.

Why not just train with tract-length augmentation and stop worrying? Augmentation is a training fix. It does not tell a user which frames of a soprano recording to trust. Nor does it remove failures caused by information loss: when $F_0 > F_1$, harmonics are too sparse to pin down the envelope, and no amount of augmentation restores it. Our ablation measures whether residual non-equivariance still tracks error after augmentation. If augmentation drives $D_\lambda$ to zero while true error stays high, the probe is shown to be blind, and we would report that.

Why not just use ensemble or dropout disagreement as the uncertainty? This is the real threat. Ensembles are cheap and widely used. Their weakness is that members trained on the same simulator share its biases, so they can agree confidently on a simulator artifact. $D_\lambda$ instead tests a property the true answer must have, whatever data the network saw. Head-to-head error prediction on the two independent physical simulators settles the matter, and we will report the outcome either way.

Why not just validate against electroglottography? As Langheinrich et al. note, electroglottography measures vocal-fold contact, not airflow. We use it only for a secondary check that glottal closure timing agrees with BITS recordings, never as flow ground truth.

Decisive Checks

The proposal stands or falls on three results. The first is predictive validity: on the Airaksinen–Story–Alku vowels and OPENGLOT, held out from training and generated by simulators other than the training simulator, per-frame $D_\lambda$ must rank true NAQ/QOQ/H1–H2 error at least as well as ensemble disagreement. Within the bootstrapped trust region, true error must be significantly lower than outside it. The second is coverage: on synthetic data, the block-multiplier bands must achieve near-nominal coverage of the true deviation curve, while independent frame resampling undercovers. The third is mechanism: the trust boundary should shift toward lower $\tilde f$ as $F_0/F_1$ approaches 1, consistent with the classical high-pitch failure. Retraining on simulator data purged of the band flagged by the simulator probe should shrink both $D_\lambda$ and true error on real-speech closure timing against BITS electroglottography. A boundary that does not move with pitch, or a $D_\lambda$ that is uncorrelated with true error, would falsify the idea.