Measuring Harmonic Bias in Formant Estimates Without Ground Truth

Formant estimators are tools that read the resonance frequencies of the vocal tract off a speech recording. When the voice is high-pitched, most of them fail in a specific, recognizable way. A voiced sound only shows spectral energy at multiples of the pitch, called harmonics. The estimated resonances therefore get pulled onto those harmonics instead of landing where the vocal tract actually resonates. Harmonic suppression weighting is a family of fixes that change which time samples the estimator trusts. They downweight the moments when the vocal folds slam shut and emphasize the "closed phase," when the tract rings freely.

These fixes are known to work on simulated voices. The trouble is that almost no one can show they work on real high-pitched voices, because the true resonances of a living singer or child cannot be measured directly. This proposal builds an evaluation that needs no ground truth. Harmonic bias leaves a fingerprint. Biased estimates cluster at multiples of the pitch, and they move when the pitch moves even though the mouth shape does not. We turn that fingerprint into a calibrated score. We check that the score ranks methods the same way true error does on physical-model simulations where the truth is known. Then we use it to report, for the first time, quantitative performance and failure profiles of these methods on natural high-pitched singing.

Core Claim

A three-part label-free score can rank formant estimators on high-pitched natural speech about as faithfully as true error ranks them on simulated speech. The first part measures excess locking of estimated formants to the harmonic grid. The second measures how much estimates drift with pitch while the vowel is held fixed. The third checks that estimates still separate vowel categories, which blocks trivial estimators from scoring well. If this holds, three questions that the literature has only answered on synthetic vowels or low-pitched read speech become answerable on real voices. Does closed-phase weighting beat the alternatives at pitches above 300 Hz? How fast does it degrade when the glottal closure markers it relies on are wrong? Does it still help once a learned tracker is available?

The locking part is simple to state. For an estimated formant $\hat F$ at a frame with pitch $F_0$, let $\delta = |\hat F/F_0 - \mathrm{round}(\hat F/F_0)|$ be the normalized distance to the nearest harmonic, which lies in $[0, 0.5]$. If estimates were unrelated to the harmonic grid, $\delta$ would be roughly uniform with mean $0.25$. The locking index is therefore

$$L = 1 - \frac{\mathbb{E}[\delta]}{0.25},$$

which is near 0 for an unbiased estimator and approaches 1 for one that sits on harmonics. The drift part is the within-vowel sensitivity $D = \partial \hat F / \partial F_0$, estimated by regression over segments where the same singer holds the same vowel at different notes. Because real vowels do shift somewhat with pitch, $D$ is scored relative to the drift of the simulator's true resonances under matched conditions, not against zero. The discriminability part is the accuracy of a fixed, simple classifier that identifies the vowel from $(\hat F_1, \hat F_2)$. It follows the observation in New signal features for robust identification of isolated vowels that pitch-induced errors are exactly what destroys vowel discrimination by formants.

Where This Sits

The harmonic suppression line supplies the methods we evaluate. Improved formant frequency estimation from high-pitched vowels by downgrading the contribution of the glottal source with weighted linear prediction introduced weighted linear prediction with an attenuated-main-excitation window (WLP-AME), which downweights samples around glottal closure. It was validated on physical-model vowels at 100–450 Hz. Quasi closed phase analysis for glottal inverse filtering built quasi-closed-phase (QCP) inverse filtering on the same window and evaluated it on 63,750 physically simulated vowels, using known glottal glottal-closure times. Time-Varying Quasi-Closed-Phase Weighted Linear Prediction Analysis of Speech for Accurate Formant Detection and Tracking extended QCP to coefficients that vary smoothly in time (TVQCP) and moved evaluation to natural speech using the Vocal Tract Resonance (VTR) corpus. None of these papers reports a quantitative result on natural voices in the high-pitch regime the methods were designed for. None measures how errors in the glottal closure markers propagate into formant error. We reuse all of them unchanged; our contribution is the measurement, not a new estimator.

A second group of papers motivates the gap. Formant Estimation and Tracking Using Deep Learning trained learned trackers on VTR. It notes that VTR labels were seeded by an automatic tracker based on linear prediction and then hand-corrected. That makes VTR partly circular for judging variants of linear prediction, and it is drawn from TIMIT read speech, which is mostly low-pitched. Glottal inverse filtering based on articulatory synthesis and deep learning trained a bidirectional recurrent network on synthesizer output. It had to deliberately underfit to transfer to real speech, and on natural recordings it could offer only qualitative evaluation. A toolkit for voice inverse filtering and parametrisation likewise falls back on visual inspection of residual ripple. Beyond bandlimited sampling of speech spectral envelope imposed by the harmonic structure of voiced sounds explains why labeling cannot fix this. At high pitch, the harmonics sample the envelope too sparsely for any spectrum reader, human or algorithm, to see the resonance directly.

The methodological stance comes from outside speech. Design and Analysis of the NIPS 2016 Review Process audited peer review, where true paper quality is unobservable, by constructing statistical proxies and checking their internal consistency. For example, it reported 90% agreement between ordinal rankings and median scores. We borrow the logic of validating a proxy by its consistency with whatever partial truth exists, not the specific statistics.

Research Program

The first step builds a calibration testbed from simulated voices. We generate vowels with known resonances from the physical modeling approach of the WLP-AME and QCP papers, and from VocalTractLab articulatory synthesis as used in Glottal inverse filtering based on articulatory synthesis and deep learning. Using two simulators with different internal assumptions guards against the circular-validation trap: an estimator that shares one simulator's modeling assumptions should not look good on both for that reason alone. On this testbed we run every estimator, compute both the true relative formant error and the label-free score, and measure the rank correlation between them across estimators and pitch bands.

The second step is a stress test designed to make the proxy fail. Trained sopranos sometimes deliberately tune their first resonance onto a harmonic. When they do, a correct estimator will show high locking and high drift, so the score would wrongly punish the right answer. This is the most serious threat to the idea. We build simulated conditions with and without such tuning, and we report how often the score misranks estimators when tuning is present. If the misranking is large, we restrict the natural-speech analysis to pitch and vowel regions where tuning is not expected, and we state that restriction plainly.

The third step applies the validated score to natural voices. High-pitched material comes from the GTSinger singing corpus, where the same vowel recurs across many notes. That recurrence is exactly what the drift measure needs. VTR's female speakers provide a mid-pitch bridge, where both labeled error and the score can be computed and compared.

The fourth step measures degradation under realistic marker error. We perturb the glottal closure markers that feed the WLP-AME and QCP windows with controlled timing jitter and with missed or spurious markers. On simulation we record the resulting change in true error; on natural singing we record the change in the score. The result is a degradation curve for each method, something the source papers acknowledge they lack.

Baselines

The naive reference point is plain linear prediction with pre-emphasis, as used in Improved formant frequency estimation from high-pitched vowels by downgrading the contribution of the glottal source with weighted linear prediction. We expect it to show the strongest harmonic locking. If our score does not rank it last at high pitch, the score is broken.

The estimators under comparison are all run with the same frame length, the same pre-emphasis, and the same prediction order, and the prediction order is also swept, because Time-Varying Quasi-Closed-Phase Weighted Linear Prediction Analysis of Speech for Accurate Formant Detection and Tracking reports that TVQCP is sensitive to it. They are robust linear prediction; discrete all-pole modeling, which fits the envelope only at the harmonic frequencies; short-time-energy weighted linear prediction; WLP-AME; QCP; TVQCP; Praat; and the DeepFormants tracker.

The strongest classical competitor on high-pitched vowels is WLP-AME. On simulated [a] it reported relative errors of 3.0% for $F_1$, 1.7% for $F_2$ and 0.5% for $F_3$, against 10.5%, 6.4% and 1.9% for robust linear prediction. On natural speech the strongest reported numbers are TVQCP-$L_1$ on VTR, with a 91.0% detection rate and 5.7% estimation error for $F_3$. DeepFormants reported mean absolute errors on VTR of 48, 83 and 109 Hz for $F_1$, $F_2$ and $F_3$, against 75, 115 and 151 Hz for Praat. We found no reported result on Lacuna for formant estimation on GTSinger or on any natural voice above 300 Hz. For those settings, our numbers are the first.

On the evaluation side, the proposal must beat two simpler proxies. The first is the correlation between formant and pitch alone. The second is VTR error used as a stand-in for high-pitch performance. "Beating" means a higher rank correlation with true error on the simulated high-pitch bands, specifically above 300 Hz. The key ablation drops each component of the score in turn. It tells us whether the discriminability term is needed to stop gaming, and whether locking alone already carries most of the signal.

Why not just…?

Why not just keep using VTR? VTR is the only natural-speech benchmark in the file with frame-level labels, and we do use it as the mid-pitch bridge. But its labels were seeded by an automatic tracker based on linear prediction, a point Formant Estimation and Tracking Using Deep Learning concedes. Its speakers are TIMIT readers with mostly low pitch. A method could top VTR and still fail at 400 Hz. Our pitch-banded comparison on simulated voices shows directly whether VTR rank predicts high-pitch rank.

Why not just evaluate on physical-model simulations, as the original papers did? Simulated voices are our calibration anchor, but they cannot tell us how methods behave on real voices. Glottal inverse filtering based on articulatory synthesis and deep learning shows how large the gap between synthetic and real speech can be. The whole point of the score is to carry simulation-validated judgments onto real recordings.

Why not just hand-label high-pitched natural speech? At high pitch the spectrum is sampled only at widely spaced harmonics, as Beyond bandlimited sampling of speech spectral envelope imposed by the harmonic structure of voiced sounds explains. A human reading a spectrogram faces the same missing information as the algorithm and is drawn to the same bright harmonic bars. Hand labels would therefore inherit the very bias we want to measure.

Why not just measure how much formants correlate with pitch? This is the real threat, because it is one component of our score and may carry most of the signal. It is exactly the ablation baseline described above. If it matches the full score's agreement with true error, we will report the simpler measure as the recommendation.

Decisive Checks

The idea succeeds if two things hold on the held-out simulator above 300 Hz. First, the score's rank correlation with true error across estimators clearly exceeds both the pitch-correlation proxy and VTR-based ranking. Second, the score does not systematically misrank estimators in the resonance-tuning condition.

It fails in one of two cases. One is if the score's ranking on simulations disagrees with true error at high pitch. The other is if the ordering of methods under the score flips between the two simulators, which would mean it rewards one simulator's assumptions instead of accuracy.

Given success, the headline results are twofold. The first is the first quantitative ranking of harmonic suppression methods on natural singing above 300 Hz. The second is marker-jitter degradation curves showing at what timing error WLP-AME and QCP lose their advantage over discrete all-pole modeling, which needs no closure markers.