Collapse Tests for Synthetic Pitch Continua

Speech scientists routinely build pitch continua: sets of synthetic stimuli in which only the fundamental frequency ($F_0$, heard as vocal pitch) changes in small steps while everything else about the voice is supposed to stay fixed. These continua are then used to locate perceptual boundaries, such as where a level Mandarin tone becomes a rising one or where a German statement starts to sound like a question. Nobody checks whether the "everything else stays fixed" assumption actually holds at each step. There is no recording of the same speaker saying the same word at every intermediate pitch to compare against, so fidelity is judged by listening or by coarse downstream scores. Computational cosmology faced the same problem, a missing ground truth, and solved it with symmetry. In simulations of scale-free universes, statistics taken at different times must collapse onto a single curve once distances are rescaled. The scales where the collapse fails show exactly where numerical resolution breaks down, with no external reference needed.

We propose the same collapse test for pitch manipulation. An ideal pitch shifter is constrained by physics. The vocal-tract filter, which carries vowel identity and timbre, must not move. The harmonic source must move by exactly the requested factor in log-frequency. Shifting by $\beta_1$ and then by $\beta_2$ must equal a single shift by $\beta_1\beta_2$. Undoing these known actions across the steps of a continuum should collapse all steps onto the original utterance. The residual, resolved by frequency band and by shift size, gives a reference-free map of where a given synthesizer can be trusted. We then test whether this map predicts where listeners hear artifacts and where published-style categorization boundaries are distorted by the synthesizer rather than the pitch.

Core Claim

A scale-resolved collapse residual, calibrated against an exactly self-similar synthetic reference, identifies the range of pitch shifts and frequency bands in which a manipulation algorithm is "converged" (its errors fall below human detectability). It does this better than distortion metrics or round-trip checks. Its main output is a certified domain of validity for each synthesizer and voice, not a single quality score. That domain lets experimenters know in advance which continuum steps are safe. The claim is falsifiable in three ways. The residual must detect and localize artifacts injected at known rates. It must flag the classic failure that round-trip tests miss. It must predict listener discrimination and boundary shifts across synthesizers better than the simpler metrics.

Relation to Prior Work

The method comes from scale-free cosmology. Quantifying resolution in cosmological N-body simulations using self-similarity measures fractional deviations of the rescaled two-point correlation function across snapshots. It uses these to find the scales converged to below 1% and shows that the lower cutoff moves as $a^{-1/2}$ in comoving coordinates. That paper also motivates our emphasis on distinguishing apparent convergence (stability under parameter changes) from physical convergence. Two follow-ups show the probe's value as a comparison tool for competing algorithms. Good and proper uses it to choose force softening. Convergence of halo statistics uses it to rank halo finders, finding that ROCKSTAR keeps self-similarity where Friends-of-Friends does not. We copy this "rank algorithms by how well they respect a known symmetry" design. It maps directly onto ranking PSOLA against neural pitch shifters.

The cosmology work has a limitation we must deal with. Its symmetry is exact because the physical model has no preferred scale. Speech has formants and a discrete harmonic comb, so it is not scale-free. Our symmetry is therefore not "the signal looks the same at all scales." It is a transformation law: the filter stays put, the source translates. The part that is not scale-free (formants) is exactly the part we require to stay still.

The representation comes from wavelet scattering. Scale dependencies and self-similar models with wavelet scattering spectra defines a testable wide-sense self-similarity. It uses scattering spectra, which are correlations of wavelet coefficients and their magnitudes across scales, and which capture intermittent, non-Gaussian structure that power spectra miss. The paper applies this only to turbulence and financial series. Wavelet Scattering on the Pitch Spiral rolls log-frequency into an octave spiral so that harmonics line up, and it separates pitch motion from timbre motion. It is used purely as a feature extractor, with no synthesis and no consistency test. We borrow the spiral's source/filter split to define the ideal action. We borrow scattering spectra to measure deviations, because PSOLA's characteristic failures are intermittent events that a smooth envelope distance averages away: buzzing at misplaced glottal pulses and phase cancellation at overlaps.

The application regime comes from the manipulation literature. Psycholinguistic continua are built with Praat PSOLA, which repositions pitch-synchronous windows to change pitch while keeping each window's shape. This is done in the 5-step Mandarin continuum of Perceptual Learning in Lexical Tone and the −4 to +6.5 semitone German stimuli of Duration and F0 interval of utterance-final intonation contours. Both treat the synthesized steps as acoustically clean. That is the assumption we test.

On the engineering side, ControlVC cascades TD-PSOLA, a quantized pitch code, HuBERT content units and a HiFi-GAN vocoder, and reports that PSOLA artifacts compound downstream. BigVGAN traces vocoder buzz to aliasing and adds anti-aliased periodic activations. Neither has an internal way to say at which shift and in which band its output stops being faithful. CFVC uses WSOLA only as a duration augmentation and does not touch $F_0$. We use these systems as the objects under test, not as methods.

Research Program

Let $x$ be a recorded utterance and $T_\beta$ a synthesizer that multiplies $F_0$ by $\beta = 2^{s/12}$ for a shift of $s$ semitones. Let $\Phi$ be a pitch-spiral scattering representation. We split it into a filter part $\Phi^{\text{env}}$ (the across-octave envelope coefficients, which ideally do not move) and a source part $\Phi^{\text{src}}$ (which ideally translates by $\log_2\beta$ octaves along log-frequency). The index $j$ denotes an octave band.

We define the equivariance residual in band $j$ as

$$D_j(\beta) = \frac{\|\Phi^{\text{env}}_j(T_\beta x)-\Phi^{\text{env}}_j(x)\| + \|\Phi^{\text{src}}_j(T_\beta x)-\tau_{\log_2\beta}\,\Phi^{\text{src}}_j(x)\|}{\|\Phi_j(x)\|} - D^{\text{floor}}_j(\beta).$$

Here $\tau_{\log_2\beta}$ is translation along log-frequency. $D^{\text{floor}}_j(\beta)$ is the residual that the analysis itself produces on an exactly scale-consistent reference. We build that reference by driving a fixed all-pole filter, estimated from $x$, with an analytic glottal pulse train. For this reference, $T_\beta$ is exact by construction, so any residual is measurement bias. The main source of that bias is that higher pitch samples the envelope more sparsely. Subtracting the floor is what separates synthesis error from analysis error. It plays the same role as the self-similar ideal in the cosmology work.

We also measure a composition residual. It checks whether two shifts in sequence equal one combined shift, and it targets the path dependence and compounding reported for cascaded systems:

$$C_j(\beta_1,\beta_2)=\frac{\|\Phi_j(T_{\beta_2}T_{\beta_1}x)-\Phi_j(T_{\beta_1\beta_2}x)\|}{\|\Phi_j(x)\|}.$$

A synthesizer's validity domain is the set of $(\beta, j)$ where $D_j$ and $C_j$ stay below a tolerance $\delta$. We set $\delta$ from a listening experiment rather than by fiat: it is the residual level at which listeners first discriminate a shifted token from the reference-synthesizer token at the same $F_0$. This is the analogue of the 1% criterion in the cosmology work, with human detectability replacing a physics tolerance.

We will build the residual pipeline, the calibrated reference synthesizer, and an artifact-injection suite. The suite adds glottal-epoch jitter at controlled rates, formant drift proportional to $\beta$, and overlap phase discontinuities. We apply the pipeline to four pitch shifters: Praat PSOLA; ControlVC used as a same-speaker converter with a constant pitch-control curve; PSOLA followed by BigVGAN resynthesis, to probe the signal-processing/vocoder interface; and naive resampling as a known-bad control.

Baselines

The naive reference points are two. The first is the identity "synthesizer," which returns $x$ unchanged. It must fail the source-translation term while passing everything else. The second is naive resampling, which shifts pitch and formants together. Both are sanity anchors: a residual that does not flag them is broken.

The strongest competing metric is the one a careful speech engineer would use today. It combines achieved-$F_0$ error, from a pitch tracker compared against the target, with a smoothed spectral-envelope distance of the Mel-cepstral distortion (MCD) type between each step and the original. We compute it per frequency band so that it has the same localization opportunity as our residual. We also compare against a round-trip test, which shifts by $\beta$ and then by $1/\beta$ and measures MCD to $x$. We compare against ASR word error rate as well, since that is how ControlVC reports quality. That paper gives 22.46% WER for ControlVC in the Pitch+Speed setting on VCTK, against 88.64% for P-LPC and 72.59% for P-AutoVC. These figures are properties of the synthesizers, not of any artifact-localization metric. We found no reported result on Lacuna for any reference-free metric that localizes pitch-manipulation artifacts by shift size or frequency band, so the comparison among metrics is new and run entirely by us.

All metrics are computed on identical stimuli from the same VCTK test speakers, using the 10-speaker test split of ControlVC. No metric is trained. The only tuned quantity for each metric is its detection threshold, chosen on held-out speakers under the same budget. Two ablations isolate our key ingredients. The first removes the floor subtraction, testing whether calibration matters. The second replaces scattering spectra with plain log-power spectra in the same collapse formula, testing whether the non-Gaussian statistics matter. We count our method as beating the envelope baseline if it wins on three things at once: detection area-under-curve for injected artifacts, band-localization accuracy, and rank correlation with listener discrimination, each by margins exceeding the bootstrap interval over speakers.

Why not just…?

Why not just do a round-trip test? Shifting up and back down is the cheapest consistency check, and it does catch non-invertible damage. But it is blind to errors that are invertible. The standard example is resampling, which moves formants with pitch and returns the original perfectly on the way back. The identity map also passes it trivially. The equivariance term exists to catch exactly these cases, and experiment (b) below demonstrates the blind spot directly.

Why not just track $F_0$ and measure envelope distance? This is the real threat. Our residual is essentially a scale-resolved, calibrated version of this idea. If a banded envelope distance with its own floor correction matches our scattering residual, then the scattering machinery is unnecessary, and we would report that. We expect a gap for two reasons. First, envelope estimators are biased by $F_0$ itself, which will register as false artifacts at high shifts unless removed; the floor ablation tests this. Second, PSOLA's buzz and phase-cancellation failures are intermittent, and Scale dependencies and self-similar models with wavelet scattering spectra argues that second-order spectra cannot see such structure. The power-spectrum ablation settles which of the two claims holds.

Why not just run listening tests at several sampling rates or shift sizes? Listening panels are the final arbiter, and we use one to set $\delta$. As a routine diagnostic, though, they are expensive and give no band localization. They also mix signal fidelity with listener strategy. Duration and F0 interval of utterance-final intonation contours found that about 25% of listeners decoded modality from 20 ms of voicing while others defaulted to "statement." Our experiments check whether a residual calibrated once against listeners transfers to new speakers and synthesizers without new panels.

Proposed Experiments

The first experiment (a) uses controlled injection on the calibrated reference synthesizer. It measures whether each metric detects artifact onset and names the correct octave band as the injection rate rises. The decisive check is whether our residual localizes formant drift and epoch jitter to the right bands while the envelope baseline smears them.

The second experiment (b) is the invertible-failure test. Resampling and the identity map must pass round-trip and fail equivariance. If they do not, the residual is not measuring what we claim.

The third experiment (c) maps validity domains for the four real synthesizers across shifts from −6 to +6 semitones. We test the concrete prediction that PSOLA-then-BigVGAN shows larger composition residuals than PSOLA alone in the high bands. That would locate the compounding reported for ControlVC in a specific frequency region.

The fourth experiment (d) is the perceptual test. We rebuild a 5-step Mandarin level-to-rising continuum after Perceptual Learning in Lexical Tone, and a German final-rise continuum spanning the ambiguous 0.5–2 semitone region of the German modality study. Each continuum is built with every synthesizer and presented to native listeners in two tasks: identification and same-different discrimination. The decisive result has two parts. First, boundary location and slope should differ across synthesizers in proportion to the residual at the steps near the boundary. Second, steps outside a synthesizer's certified domain should be the ones listeners discriminate from reference tokens.

If boundaries do not move with residuals, then synthesizer artifacts are perceptually irrelevant to categorization. That would be a useful negative result, suggesting PSOLA-based continua in the existing literature are safe. If boundaries do move, published boundary estimates carry a synthesizer-dependent bias that our map can predict before any listener is run.