Harmonic Staircases: Testing Whether Vowel Perception at High Pitch Tracks Discrete Harmonics

A voiced vowel is a periodic sound. Its spectrum is therefore not a continuous curve but a comb of harmonics spaced $F_0$ apart, where $F_0$ is the fundamental frequency, heard as pitch. Each harmonic's height samples the vocal-tract resonance curve, and the peaks of that curve, called formants, are what phonetics textbooks say carry vowel identity. At speaking pitch the comb is dense, so the curve is well sampled. In soprano or opera singing, $F_0$ can reach 800–900 Hz, above where the first formant of /i/ or /u/ sits (around 250–300 Hz). Here the comb skips that formant entirely. Listeners still identify these vowels: native listeners recognised /i/ above 80% correct up to 859 Hz in a Cantonese opera corpus (Intelligibility of high-pitched vowel sounds…). Nobody has a quantitative account of what the auditory system computes in this regime. The field's current language, a "switch from formant cues to whole-spectrum cues," predicts no specific curve.

This note proposes a sharp test between two accounts. In the first, the listener reconstructs a smooth envelope and reads it. In the second, the listener reads the finite set of sampled harmonics directly. We borrow one idea from dimensional reduction in physics. When a field lives on a small circle of radius $R$, the low-dimensional observer sees a discrete tower of modes. Observables of the reduced theory can become non-smooth at the parameter values where a mode crosses a critical threshold. With $R \sim 1/F_0$, the harmonic comb is exactly such a tower. The sampling account then predicts that vowel-identification curves, measured as a function of $F_0$ with the vocal tract held fixed, form a staircase: they are piecewise smooth, with breakpoints at $F_0 = F_k/n$, where a harmonic $n$ crosses formant $k$. The reconstruction account predicts smooth curves. That difference can be measured.

Core Claim

The physics supplies a lens and an analysis tool, not a mechanism. The tool comes from Discontinuities of free theories on AdS2. That paper shows that when theories on $AdS_2 \times S^1$ are reduced to $AdS_2$, varying the ratio of radii produces a discrete sequence of non-smooth points in the reduced observables, one each time a mode crosses a critical mass. It also distinguishes two kinds of non-smooth point: kinks (a change of slope, for bosons at the Breitenlohner–Freedman bound) and jumps (a step, for fermions at zero mass). We use that result for two things. It sets the expected geometry of the breakpoints: a countable set whose locations are fixed by the ratio of an internal scale to a mode spacing. It also supplies the diagnostic we will apply to perceptual data: locate the non-smooth points, then classify each one as a kink or a jump.

The testable claim is this. Write $P_v(F_0; E)$ for the probability that listeners report vowel $v$ for a fixed resonance curve $E$ excited at pitch $F_0$. The claim is that $P_v$ is smooth between the breakpoint set

$$\mathcal{B}(E) = \{\, F_k/n : k = 1,2,3;\ n = 1,2,\dots \,\},$$

where $F_k$ are the formant frequencies of $E$ and $n$ indexes harmonics. The claim is also that non-smooth points concentrate near $\mathcal{B}(E)$ more than chance placement would allow. A secondary, riskier claim is that crossings of $F_1$ by the lowest harmonics ($n = 1, 2$) produce jumps rather than kinks. The reasoning is that in this regime a single resolved harmonic is the only evidence for $F_1$.

This claim is new in a specific sense. Formant Pattern and Spectral Shape Ambiguity of Vowel Sounds shows that the same formant pattern is heard as different vowels at different $F_0$. That motivates the claim, but it is demonstrated with isolated examples rather than finely sampled curves, and it proposes no rule for where the percept changes. Relative importance of formant and whole-spectral cues and An effect of formant amplitude in vowel perception show that envelope shape and the relative amplitudes of formants matter. Both, however, fix $F_0$ at 125 Hz and $F_1$ at 250 Hz, so they cannot see $F_0$-dependent structure at all. The staircase hypothesis is the missing $F_0$ axis for their results.

Research Program

The program has three parts: a listening experiment, two computational models, and a re-analysis of existing corpora.

Listening experiment. We synthesise vowels from fixed resonance curves with a cascade formant synthesizer, the same family used by Ito et al. The curves are drawn from their /i/–/u/ matrix and extended to the German and Cantonese vowel sets. We step $F_0$ from 200 to 900 Hz in 5 Hz increments, which places many samples between adjacent predicted breakpoints. Loudness is equalised so that overall level cannot cue identity.

Listeners do forced-choice identification. The cohort is roughly 25 native German listeners and roughly 25 native Cantonese listeners, far more than the 6–8 listeners in the Klatt studies. Trials are interleaved and randomised to limit context effects. A control set uses the same curves with noise excitation instead of a harmonic comb. The noise has no $F_0$, so its identification curve should be flat.

Two models. Both read the same auditory front end, an excitation pattern on a Bark or ERB frequency scale. Both are trained on the same data.

The harmonic-tower model sees only the levels at resolved harmonic positions, plus $F_0$ as an explicit context variable. $F_0$ is included because Maurer's results require that the listener can use it. Its readout is a small network. Its breakpoints are not hand-placed; they arise from where harmonics enter and leave auditory filters.

The envelope-reconstruction model first interpolates a smooth curve through the harmonic levels, using cepstral smoothing or a true-envelope estimate. Only then does it classify.

These two models differ in exactly one ingredient: whether the classifier can see the envelope between harmonics or only at them. Each model predicts the full identification surface $P_v(F_0; E)$, which we compare to the human data.

Corpus re-analysis. We fit the same models on the Zurich Corpus of Vowel and Voice Quality, about 34,600 sustained German vowels spanning a wide $F_0$ range and several phonation types. We then test on held-out high-$F_0$ recordings. We also test on the 172 Cantonese opera sounds, where the target is to predict the human per-vowel upper limits for identification above 80% correct: /y/ 692, /œ/ 765, /a/ 823, /ɔ/ 845, /u/ 855, /i/ 859 Hz.

Analysis. Following the physics diagnostic, we fit each human curve with segmented regression that allows both kinks and steps. Breakpoints are chosen by cross-validated likelihood, and the procedure never sees $\mathcal{B}(E)$. We then measure the distance from each detected breakpoint to the nearest element of $\mathcal{B}(E)$. This is compared against a null in which the same number of breakpoints is placed uniformly at random over the $F_0$ range.

Baselines

Naive reference: formant-frequency classifier. The reference point is a Gaussian classifier on $(F_1, F_2, F_3)$ read directly from the synthesis parameters. It embodies the classical source-filter view that vowel identity depends on formant locations alone. It predicts perfectly flat curves over $F_0$. Beating it means capturing any $F_0$ dependence at all. If human curves are flat, the whole proposal fails immediately.

Envelope-smoothing classifier. This baseline uses standard cepstral or linear-predictive smoothing followed by a classifier, trained on Zurich with the same tuning budget as our models. It is the strongest simple competitor because it is what speech technology actually does. It is also the natural null for the "whole-spectrum cue" story in Relative importance of formant and whole-spectral cues. The envelope-reconstruction model above is a tuned member of this family.

Pitch-dependent recognition. PISAR (Pitch synchronous spectral analysis…) analyses each pitch period separately and trains a separate recogniser for each of 15 pitch bins. It is the strongest published method that explicitly conditions on $F_0$. It reports 59.1% voiced-phoneme recognition on clean TIMIT against 43.4% for standard cepstral analysis, and 42.2% against 39.9% in car noise at 5 dB SNR. We reimplement it on Zurich and the synthetic stimuli with our bins matched to its 15 regions. Because its bins produce steps at arbitrary bin edges, it is a direct test of whether our breakpoints are merely a binning artefact. No reported result was found on Lacuna for PISAR, or any model, on the Cantonese opera corpus or on vowels above about 600 Hz.

Learned disentangled representation. This baseline is a self-supervised speech encoder with a pitch-invariance objective, in the spirit of Towards the Next Frontier in Speech Representation Learning Using Disentanglement. We fine-tune it on Zurich with the same data and compute as the other models.

Key ablation. The key ablation is the tower model versus the envelope-reconstruction model, holding the front end, training data and readout fixed. Beating a baseline means two things. First, a better held-out log-likelihood on the human identification surface. Second, on the opera sounds, a higher rank correlation with the six human upper-$F_0$ limits and a smaller absolute error in Hz.

Why not just…?

Why not just smooth the spectrum and classify, as speech recognisers do? Smoothing assumes the envelope between harmonics is recoverable. At 800 Hz spacing, it is not. Maurer's examples show that a static envelope is ambiguous across $F_0$ (Formant Pattern and Spectral Shape Ambiguity), and cepstral analysis already loses 15.7 points to pitch-aware analysis on TIMIT in the PISAR comparison. Still, a good envelope estimator plus $F_0$ context could match human curves. The key ablation settles this. If the envelope-reconstruction model fits as well as the tower model and human curves are smooth, the staircase hypothesis is wrong and we will report that.

Why not just train a big neural network with pitch disentanglement? It may predict the curves well. But a network that is invariant to pitch by construction is the wrong target, since human percepts demonstrably change with $F_0$. Neural formant synthesis also degrades under large parameter scaling (Speaker-independent neural formant synthesis). We include the network as a baseline, and we probe whether its predicted curves show breakpoints at $\mathcal{B}(E)$. If it matches humans and shows them, that supports the sampling claim through a different route.

Why not just call this sampling theory, without the physics? This is a fair point. The breakpoint locations follow from elementary sampling arguments, and the harmonic-tower model needs no physics to state. What the dimensional-reduction lens adds is modest but useful. It frames the question as whether the listener's "effective theory" depends smoothly on the internal scale $1/F_0$. And it supplies a classification of non-smooth points, kinks versus steps, from the AdS₂ analysis, which turns a vague "categorical shift" into two distinguishable curve shapes. We do not claim the auditory system implements the reduction.

Why not just use PCA on spectra? Linear projections of the spectrum vary smoothly with $F_0$ unless the input itself changes discontinuously. They are therefore subsumed by the envelope baseline. Thaine and Penn's SVD result (Vowel and Consonant Classification through Spectral Decomposition) concerns written text, not sound, and does not bear on this question.

Decisive Checks

The first and decisive check is on the human data. Segmented regression must prefer piecewise-smooth fits over smooth fits, and detected breakpoints must lie closer to $\mathcal{B}(E)$ than the random-placement null allows. The noise-excited control must show flat curves. If breakpoints appear but do not align with $F_k/n$, the tower model is wrong in its details even if the sampling idea survives.

The second check is the key ablation. The tower model should beat envelope reconstruction on held-out human surfaces, and it should reproduce the ordering of the opera upper limits, with /y/ lowest and /i/ and /u/ highest.

The third, riskier check is the kink-versus-step prediction: lowest-harmonic crossings of $F_1$ should produce steps, and higher crossings should produce kinks.

Finally, PISAR must not explain the breakpoints. Its bin edges are fixed and independent of $E$, whereas ours move with $E$. Showing that breakpoints shift when formants shift, at the same $F_0$, separates the two accounts.