Where Pitch Normalization Breaks: Register-Resolved Tests Borrowed from the 12% Rule

Voice conversion systems copy the melody of one voice onto another by first "whitening" the pitch, meaning they subtract the source speaker's average pitch and divide by its spread, and then rescaling the result to the target speaker's average and spread. This works well when the two voices have similar ranges. It degrades on large shifts, such as male to female or speech to singing, and the field usually describes that degradation as a smooth cost of distance. We propose that the cost is not smooth. We hypothesize that it concentrates in the frames where the remapped pitch crosses into a different physiological register of the target voice, for example from chest voice into falsetto. Utterance-level quality scores average these frames away. The plan has two parts. First, a diagnostic that measures conversion quality separately for each register "channel" relative to a self-conversion reference, so the failure can be located. Second, a register-anchored normalization that keeps separate pitch statistics for each register and tells the decoder which register it is producing.

The way of measuring comes from heavy-quarkonium physics. There, the "12% rule" predicts that every hadronic decay of the $\psi(2S)$ meson scales relative to the $J/\psi$ in proportion to their leptonic decay rates. The leptonic ratio acts as a single global normalization that cancels the bound-state wavefunction. That normalization holds for multi-pion final states, as Measurement of the branching fractions of ψ(2S)→3(π+π−) and J/ψ→2(π+π−) showed. It fails by about a factor of eight in the specific two-body channel $\rho\pi$, where Observation of 1−0− Final States from ψ(2S) decays and e+e− annihilation measured $Q=(1.5\pm0.5)\times10^{-2}$ against the expected $12\times10^{-2}$. The anomaly was found only because experimenters reported ratios channel by channel and subtracted a continuum background measured off-resonance. Speech evaluation does neither. We are not claiming to explain the $\rho\pi$ puzzle. We borrow the measurement discipline, and we test whether speech has an analogous "channel-selective" failure of a global normalization.

Core Claim

Global affine pitch normalization, whether z-scored in Hz or in log-pitch, fails mainly at register crossings rather than in proportion to the size of the pitch shift. The specific prediction is about where quality loss sits after self-conversion artifacts are subtracted. Most of the excess degradation in large-shift conversion should occur in frames where the remapped contour lands in a register the source frame was not in. A register indicator should explain that degradation beyond what semitone distance explains. If this holds, the remedy is to make normalization and decoding register-aware. Smoother nonlinear pitch maps would not be enough, because they still push modal-voice notes across the target's register break without the decoder knowing.

The claim is falsifiable. If degradation grows smoothly with semitone shift, or if crossing frames carry no excess error once shift is controlled for, the hypothesis is wrong. In that case simple quantile mapping, described below, is the right fix, and the proposal reduces to a negative diagnostic result.

Prior Work and Its Roles

Three works supply the method we build on. They also illustrate the global-normalization assumption we test.

StreamVC injects z-score-whitened pitch, together with causal content units, into a decoder conditioned on the target speaker. It reaches 70.8 ms latency on a phone. Its normalization is a single affine map per speaker, and we keep that map as our main baseline.

Neekhara et al. and Liu et al. use the same shift-and-scale re-projection for style transfer. Lee subtracts source mean pitch so that extreme expressive styles stay within the target's comfort zone. All of them assume one set of statistics per speaker. We relax that to one set per speaker per register.

PitchFlow guides a flow-matching model, a generator that turns noise into audio along a learned path, toward a requested pitch contour. It notes that octave errors in pitch tracking destabilize generation. This constrains our design: register labels must not be inferred from pitch alone, or tracking errors would masquerade as register crossings.

The motivating evidence comes from Everyone-Can-Sing. It does zero-shot singing synthesis from speech references and reports that cross-gender conversion with large pitch shifts brings a "slight drop in quality." We read "slight" as possibly an artifact of averaging over whole utterances. That reading parallels how the inclusive multi-pion channels in the BESII measurement obey the rule while specific exclusive channels do not.

On the physics side, Comprehensive calculations on the OZI-forbidden nonleptonic decays of orthoquarkonia shows that the standard factorized theory gets $J/\psi\to\pi^+\pi^-$ right but underpredicts $\rho\pi$ by an order of magnitude. This is a cautionary example: a factorization that is correct on average can be badly wrong in a structurally distinct channel. It motivates looking for the structurally distinct channel in speech, the register boundary, rather than tuning the global map.

Research Program

The data backbone is GTSinger, described in GTSinger: A Global Multi-Technique Singing Corpus. It suits this question for three reasons. It contains 80.59 hours from 20 professional singers. Each lyric is sung twice, once naturally and once with a technique such as falsetto or mixed voice. It also has phoneme-level technique annotations and 16.16 hours of paired speech. These annotations give us ground-truth register labels, which are independent of the pitch tracker. We supplement GTSinger with LibriTTS-R for speech-to-speech conversion across genders.

The diagnostic defines a per-channel ratio, by analogy with $Q$. For a channel $c$, such as "modal to modal," "modal to falsetto," or "speech to mixed voice," we compute

$$Q_c=\frac{D_c^{\text{conv}}-D_c^{\text{self}}}{\bar D^{\text{conv}}-\bar D^{\text{self}}}.$$

Here $D_c^{\text{conv}}$ is the mean frame-level degradation in channel $c$ after conversion to another speaker. $D_c^{\text{self}}$ is the same quantity when converting a speaker to itself; this serves as the analog of continuum subtraction and removes vocoder and decoder artifacts that have nothing to do with normalization. The bars denote the inclusive average over all frames. Degradation combines three components: pitch error against the intended contour, an octave-error rate, and a frame-level naturalness proxy that we validate against short-segment listening tests. Under the global-normalization hypothesis, $Q_c\approx1$ in every channel. Our hypothesis predicts $Q_c\gg1$ for crossing channels.

Register labels at inference time come from a frame-level register classifier trained on GTSinger's technique annotations. GTSinger reports F1 of 0.99 for breathy voice; we found no reported falsetto figure, so we will measure it.

The remedy, register-anchored normalization, replaces one $(\mu,\sigma)$ pair per speaker with one pair per speaker and register $r$, computed in log-pitch. A source frame in register $r$ is whitened with the source's $r$-statistics. It is then remapped with the target's statistics for the register the target would use at the resulting pitch, and that register label is passed to the decoder as conditioning.

The hard case is a zero-shot target heard only speaking, which gives no falsetto statistics. For this case we fit a population regression from modal statistics to falsetto statistics across GTSinger singers. Twenty singers is a small sample, and we treat that regression as the most fragile part of the system and report its error separately.

Baselines

The naive reference is no normalization: the absolute source pitch is copied to the target. This sets the floor, and it is useful mainly to show that the self-conversion subtraction behaves sensibly.

The strongest existing method is StreamVC-style global z-score whitening with affine remapping. We reimplement it on the same content units, decoder, data, and step budget as our system. The two systems differ only in the pitch normalization module and the register conditioning. The best published reference figures come from StreamVC on LibriTTS: pitch Pearson correlation 0.842, word error rate 6.22%, and automatic naturalness score 4.07. For large-shift singing, Everyone-Can-Sing reports quality MOS 3.98 and similarity MOS 3.78 from speech references. We found no reported result on Lacuna for any method on cross-register conversion within GTSinger, so absolute comparisons there will come from our own runs. To beat this baseline, we must reduce excess degradation in crossing channels, meaning $Q_c$ moves toward 1, with confidence intervals from bootstrapping over singers. We must also not lose more than about 0.01 in pitch correlation or 0.3 points of word error rate on within-register LibriTTS conversion.

A second strong competitor is log-pitch z-scoring combined with quantile mapping. Quantile mapping replaces the affine remap with a monotone map that matches the full source and target pitch distributions. It gets the same tuning budget.

Two ablations isolate the key ingredient. The first passes the register label to the decoder but keeps global statistics. The second uses per-register statistics but gives the decoder no label. The second ablation separates "the statistics were wrong" from "the decoder needed to know the register." We also run an oracle that uses GTSinger's ground-truth register labels instead of the classifier, which bounds the cost of classification errors.

Why not just…?

Why not just use quantile mapping instead of linear z-scores? This is the most serious threat. A monotone distribution-matching map fixes range mismatch at no architectural cost. It is still a single smooth map, though. It sends a source chest-voice note to wherever the target's distribution puts it, including above the target's register break, and the decoder never learns that the target would sing that note differently. The diagnostic settles the question directly. If quantile mapping drives crossing-channel $Q_c$ to about 1, the register mechanism is unnecessary and we will say so.

Why not just train a bigger generative model and let it learn the mapping? PitchFlow reports that even large flow-matching and VALL-E systems needed explicit guidance to follow melody and hold identity: similarity MOS was 3.88 unguided versus 4.02 with guidance, on training data of about 6 million recordings. Singing data with register labels amounts to tens of hours. We will still include the strongest available large zero-shot system as a reference and compute its $Q_c$. If a large model shows no channel-selective failure, our hypothesis loses much of its practical relevance.

Why not just condition on technique labels and skip the normalization change? That is exactly the first ablation. If it matches the full system, then register-specific statistics add nothing beyond the label, and the contribution narrows to conditioning.

Decisive Checks

The first check is the diagnostic regression. On held-out GTSinger singers and on cross-gender LibriTTS-R pairs, we regress subtracted frame degradation on absolute semitone shift and on a register-crossing indicator, computed from oracle labels so that tracker errors cannot create spurious crossings. The hypothesis survives only if the crossing indicator carries significant explanatory weight once shift is controlled for, and if global-baseline $Q_c$ for crossing channels clearly exceeds 1 while within-register channels sit near 1.

The second check is the remedy comparison. Register-anchored normalization must lower crossing-channel $Q_c$ relative to both global z-scoring and quantile mapping, while staying within the stated tolerances on within-register LibriTTS conversion. We confirm the result with listening tests on crossing segments presented in isolation, so that averaging over the whole utterance cannot hide the effect.

The third check is the zero-shot falsetto case. We report the error of the modal-to-falsetto statistics regression, and how much of the gain survives when the target was heard only speaking.

If the first check fails, we publish it as a negative result. That result would still be useful, because it would show that the large-shift degradation reported in Everyone-Can-Sing is a distance effect best handled by distribution matching.

References

  1. Measurement of the branching fractions of<mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="inline"><mml:mi>ψ</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mn>2</mml:mn><mml:mi>S</mml:mi><mml:mo stretchy="false">)</mml:mo><mml:mo>→</mml:mo><mml:mn>3</mml:mn><mml:mo stretchy="false">(</mml:mo><mml:msup><mml:mi>π</mml:mi><mml:mo>+</mml:mo></mml:msup><mml:msup><mml:mi>π</mml:mi><mml:mo>−</mml:mo></mml:msup><mml:mo stretchy="false">)</mml:mo></mml:math>and<mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="inline"><mml:mi>J</mml:mi><mml:mo>/</mml:mo><mml:mi>ψ</mml:mi><mml:mo>→</mml:mo><mml:mn>2</mml:mn><mml:mo stretchy="false">(</mml:mo><mml:msup><mml:mi>π</mml:mi><mml:mo>+</mml:mo></mml:msup><mml:msup><mml:mi>π</mml:mi><mml:mo>−</mml:mo></mml:msup><mml:mo stretchy="false">)</mml:mo></mml:math>
  2. Observation of<mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="inline"><mml:msup><mml:mn>1</mml:mn><mml:mo>−</mml:mo></mml:msup><mml:msup><mml:mn>0</mml:mn><mml:mo>−</mml:mo></mml:msup></mml:math>Final States from<mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="inline"><mml:mi>ψ</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mn>2</mml:mn><mml:mi>S</mml:mi><mml:mo stretchy="false">)</mml:mo></mml:math>Decays and<mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="inline"><mml:msup><mml:mi>e</mml:mi><mml:mo>+</mml:mo></mml:msup><mml:msup><mml:mi>e</mml:mi><mml:mo>−</mml:mo></mml:msup></mml:math>Annihilation
  3. STREAMVC: Real-Time Low-Latency Voice Conversion
  4. Expressive Neural Voice Cloning
  5. DSE-TTS: Dual Speaker Embedding for Cross-Lingual Text-to-Speech
  6. Robust and Fine-grained Prosody Control of End-to-end Speech Synthesis
  7. PitchFlow: adding pitch control to a Flow-matching based TTS model
  8. Everyone-Can-Sing: Zero-Shot Singing Voice Synthesis and Conversion with Speech Reference
  9. Comprehensive calculations on the OZI-forbidden nonleptonic decays of orthoquarkonia
  10. GTSinger: A Global Multi-Technique Singing Corpus with Realistic Music Scores for All Singing Tasks