Voice conversion makes one person's recorded speech sound like another speaker while keeping the words intact. Its objective scores are measurements, and like any measurement they carry systematic bias from the instrument. The instrument here is the analysis pipeline, which slices audio into short overlapping frames, estimates a smooth spectral envelope per frame with a vocoder (a model that splits speech into envelope, pitch, and excitation), and time-aligns converted and reference speech before computing a distance. Every one of these steps is a choice of grid and scheme. The field routinely reports mel-cepstral distortion, a frame-averaged distance between spectral envelopes, at one fixed setting (5 ms frame shift, one vocoder) and treats the number as a property of the conversion system. We propose to borrow two habits from precision physics. From lattice field theory we take the practice of computing a quantity at several grid spacings and with more than one discretization, and trusting only what survives the limit. From reactor-neutrino experiments we take the practice of forming ratios against a reference that shares the same instrument biases, so those biases cancel.
The concrete payoff is an objective score whose system rankings are stable across analysis choices, and a test of a specific suspicion. Part of the objective penalty that conversion systems show on cross-gender pairs may come from how envelope estimation behaves at high pitch, not from worse conversion. Listeners do report a real cross-gender drop: The Voice Conversion Challenge 2018: database and results finds an average 0.3 MOS loss for traditional systems. The question is how much of the objective gap tracks that drop and how much is measurement artifact.
Core Claim
We claim that a voice-conversion distance should be treated as an observable with a continuum limit and a scheme dependence. Its instrument-shared bias can be removed by differencing against the natural target passed through the identical analysis–resynthesis chain. After both corrections, the score should rank systems in closer agreement with listener judgments than raw mel-cepstral distortion does. It should also change how much of the objective cross-gender gap remains.
The methodological template comes from High-precision scale setting in lattice QCD. That paper computes the gradient-flow scale $w_0$ at several lattice spacings with two unrelated fermion discretizations. It trusts the result only because both extrapolate to the same continuum value. We reuse exactly this logic, with WORLD-style and STRAIGHT-style analysis playing the role of the two actions. Improved actions for the two-dimensional σ-model shows that the size of grid artifacts depends strongly on the discretization chosen, with the plaquette action removing roughly half the scaling error in $N=3$ simulations. That result motivates the concern that analysis-scheme choice alone can reorder conversion systems. The ratio idea comes from Model-independent \bar{\nu}_e short-baseline oscillations from reactor spectral ratios, which cancels poorly known flux and efficiency factors by dividing spectra measured with shared systematics. It also comes from Daya Bay's independent nH-channel measurement, whose value lies in having different backgrounds from the main channel.
One piece of the analogy does not transfer, and we say so plainly. In lattice QCD, extrapolation exists because fine grids are expensive. In speech, running at a 1 ms frame shift costs almost nothing. Our frame-shift extrapolation is therefore a consistency check on alignment and delta-feature artifacts, not a cost-saving device. The scheme-universality test and the ratio are the substantive parts.
Research Program
Let $h$ be the frame shift and $s$ the analysis scheme, meaning the vocoder and window choice. Let $D_s(h)$ be the distance between converted and natural target speech, computed after dynamic time warping, the standard frame-matching alignment step. Guided by the Symanzik picture, in which leading grid artifacts enter as powers of the spacing, we fit
$$D_s(h) = D_s^{(0)} + c_s\, h^{p} + \dots,$$
where $D_s^{(0)}$ is the continuum-limit distance, $c_s$ is a scheme-dependent artifact coefficient, and $p$ is fitted. The fitted $p$ is expected near 1 or 2. The value 1 would come from alignment granularity, and 2 from the second-order truncation error of the standard central-difference velocity and acceleration features used in trajectory generation. We use frame shifts of 10, 5, 2.5, and 1.25 ms.
The ratio-corrected score subtracts the same quantity measured for the natural target utterance resynthesized through the same scheme:
$$\tilde D_s = D_s^{(0)}[\text{converted}] - D_s^{(0)}[\text{target resynthesized}].$$
This subtraction removes bias that is common to any signal passing through the vocoder and alignment. The second term depends only on the target speaker and scheme. That property is exactly what makes it an instrument calibration rather than a property of the system under test.
Universality is then an empirical test. If $\tilde D_s$ ranks systems identically under two schemes while raw $D_s(5\,\text{ms})$ does not, the correction is doing work. If neither corrected nor raw scores agree across schemes, the envelope estimate itself is scheme-dominated, and objective comparison across vocoders is unsafe at the precision people report. Either outcome is informative.
We will evaluate on submitted converted speech from the Voice Conversion Challenges, which comes with listener scores. We will use VCC 2018, with its parallel Hub and non-parallel Spoke tasks, 35 evaluation sentences, and 267 listeners. We will also use VCC 2016, with 16 speaker pairs and 200 listeners, analyzed in Analysis of the Voice Conversion Challenge 2016 Evaluation Results. We rely on the challenge releases of system outputs. If only a subset is recoverable, we additionally train our own controlled systems on the same data. These include the open-source GMM baseline sprocket and a DIFFVC-style differential-filtering system following The NU-NAIST system. For these we collect a small listening test so every system has matched human scores.
Our own systems also give a second, conversion-side use of the corrected metric. For those systems we can vary the frame shift at training and generation time. We can also replace the central-difference delta windows in trajectory generation with a higher-order (Symanzik-style) stencil. The corrected score then tells us whether either change matters at all once measurement artifacts are removed. The recursive trajectory method of Low-delay voice conversion based on maximum likelihood estimation of spectral parameter trajectory fixes $h = 5$ ms and never varies it. We will re-run it on its BTOS body-conducted corpus at the four shifts to see whether its reported delay–quality curve depends on $h$.
Baselines
The naive reference point is raw mel-cepstral distortion at a 5 ms shift with a single vocoder, computed exactly as the challenge-era systems report it. That paper's low-delay results on BTOS are reported this way, as mel-cepstral distortion curves versus delay, but without a scalar we can quote here. Our corrected score must beat this reference in rank agreement with listener MOS and similarity over the same systems and utterances. We will compute it on identical alignments and sentences.
The strongest existing objective alternative within the file is word error rate from an automatic recognizer. VCC 2018 reports a strong negative correlation between WER and naturalness, but no reported correlation value was found on Lacuna for either WER or mel-cepstral distortion against MOS. We will therefore compute both correlations ourselves on the same system set, as the bar to clear.
The human anchors come from the challenge papers. For VCC 2018 Hub, natural speech scored 4.5 MOS, the best system N10 scored 4.1, and sprocket scored 3.2. For VCC 2016, the NU-NAIST System J reached about 74% similarity. These fix the ranking our score must reproduce, in particular separating N10 from sprocket and preserving the reported intra-gender versus cross-gender ordering.
Two ablations isolate the ingredients: the continuum extrapolation without the ratio, and the ratio at fixed $h$ without the extrapolation. All variants share the same utterances, alignments, and bootstrap resampling over sentences and listeners. Beating a baseline means a higher Spearman correlation whose bootstrap interval excludes the baseline's value. A pre-registered minimum difference of 0.1 in rank correlation counts as practically meaningful.
Why not just…?
Why not just compute everything at a 1 ms shift and skip extrapolation? This is a real threat to the extrapolation component, and we expect it may win. If $D_s(1.25\,\text{ms})$ and $D_s^{(0)}$ agree within bootstrap error for all systems, we will report that the frame-shift limit is already reached cheaply and drop extrapolation from the recommendation. The scheme-universality test and the ratio do not depend on this outcome. Shrinking the shift does nothing about the vocoder or window, which play the role of the lattice action.
Why not just normalize by the analysis–synthesis upper bound, as people informally do? This is essentially our ratio term, and we treat it as a competitor rather than a contribution. The ablation "ratio only, fixed $h$, single scheme" is exactly this practice. Our claim adds only that the calibration term must be computed per target speaker and per scheme, and that it must be checked across schemes. If the informal version matches the full method, we say so.
Why not just train a neural MOS predictor on listener data? A learned predictor may correlate well with listeners inside its training distribution. However, it gives no account of which part of a score is instrument bias, so it cannot answer the cross-gender question. We found no reported result on Lacuna for a learned MOS predictor on VCC data. Such a model trained on VCC 2016 and tested on VCC 2018 would be a fair extra comparison if one is available.
Proposed Experiments
The first decisive check is universality. For each challenge system, we compute raw and corrected scores under two analysis schemes and measure the rank correlation between the two schemes' system orderings. The hypothesis fails if correction does not raise cross-scheme agreement.
The second check is listener agreement. Using the same systems, we compare Spearman correlations with MOS and with similarity for raw distortion, WER, the two ablations, and the full score, reporting Hub and Spoke separately.
The third check is the cross-gender decomposition. We compare the objective intra- versus cross-gender gap before and after correction with the listener gap of about 0.3 MOS. Suppose the corrected gap shrinks while its correlation with the listener gap rises. That would show that part of the raw objective penalty is a pitch-dependent measurement artifact that the ratio removes. Suppose instead the gap is unchanged. Then the objective penalty reflects conversion quality, which is also worth knowing.
The fourth check uses our own sprocket and DIFFVC-style systems. We train at the four shifts, with and without higher-order delta stencils, and ask whether any difference survives correction and shows up in a small listening test. The expected, honest outcome is that shift and stencil effects are small. Showing that with a trustworthy metric closes a question the trajectory-generation literature has left implicit.
References
- The Voice Conversion Challenge 2018: database and results
- High-precision scale setting in lattice QCD
- Improved actions for the two-dimensional σ-model
- Model-independent <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" altimg="si1.gif" overflow="scroll"><mml:msub><mml:mrow><mml:mover accent="true"><mml:mrow><mml:mi>ν</mml:mi></mml:mrow><mml:mrow><mml:mo stretchy="false">¯</mml:mo></mml:mrow></mml:mover></mml:mrow><mml:mrow><mml:mi>e</mml:mi></mml:mrow></mml:msub></mml:math> short-baseline oscillations from reactor spectral ratios
- Independent measurement of the neutrino mixing angle<mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="inline"><mml:mrow><mml:msub><mml:mrow><mml:mi>θ</mml:mi></mml:mrow><mml:mrow><mml:mn>13</mml:mn></mml:mrow></mml:msub></mml:mrow></mml:math>via neutron capture on hydrogen at Daya Bay
- The Voice Conversion Challenge 2016
- Analysis of the Voice Conversion Challenge 2016 Evaluation Results
- The NU-NAIST Voice Conversion System for the Voice Conversion Challenge 2016
- Low-delay voice conversion based on maximum likelihood estimation of spectral parameter trajectory