Voice conversion systems separate what is said from who says it. They do this by squeezing speech through a narrow "content" channel, either a small hidden layer or a small codebook of discrete codes, and handing the decoder the speaker's identity separately. The width of that channel is a knife-edge. If it is slightly too wide, the source speaker's voice leaks through. If it is slightly too narrow, words become unintelligible. Today the width is found by grid search, with full listening tests at each setting. AVQVC is a vector-quantization voice converter, meaning it replaces each frame's features with the nearest entry in a learned codebook. Its authors report that 512 codes separate timbre from content cleanly, while 1024 codes let timbre leak "immediately." That abruptness looks less like gradual degradation and more like a phase transition.
Physicists studying measurement-induced phase transitions face a similar problem: locating a sharp transition when no obvious quantity is known to change across it. These are transitions in quantum circuits where random scrambling competes with repeated local measurements. They do not look for a local order parameter. They track how much information two regions of a system share, and they read the transition off sharp peaks in the derivative of that shared information as the control knob is turned. This note borrows that diagnostic, not the quantum physics. In a well-disentangled content stream, two distant segments of the same utterance should share almost nothing, because they carry different words. If speaker identity leaks into the content stream, every frame carries the same voice, and distant segments become correlated. That long-range sharing can be estimated with no labels at all, using only which segments came from the same recording. The proposal is to use it as a label-free order parameter for leakage, locate the leakage transition from its derivative, and test whether the best voice conversion sits at that transition.
Core Claim
Speaker leakage in a content bottleneck behaves like the onset of long-range order. A useful analogy is a magnet: below a critical temperature, spins far apart point the same way. Here, below a critical capacity the content codes are "disordered": correlations between distant frames decay to the small level that phonotactics and lexical context can explain. Above it, a component shared by every frame appears, namely the speaker's voice, and correlations between distant frames plateau.
This order parameter can be estimated from unlabeled audio. Its derivative with respect to capacity should peak sharply at a critical capacity $K_c$. We make three falsifiable predictions. First, on the AVQVC architecture trained on VCTK, the peak falls between 512 and 1024 codes, where the grid search found leakage. Second, $K_c$ found this way matches the threshold found by a labeled speaker probe. Third, the best trade-off between speaker similarity and intelligibility sits at or just below $K_c$. The third prediction is the speech analogue of the finding by Measurement-Induced Criticality is Tomographically Optimal. That paper shows that the cost of reconstructing a quantum state from measurements is minimized exactly at the transition. That is a claim about quantum circuits, and we treat its speech counterpart as a hypothesis to test, not a consequence.
Concretely, let $Z_A$ and $Z_B$ be the content codes of two non-overlapping segments of one utterance, separated by $s$ frames. Let $K$ be the bottleneck capacity (codebook size or bits per frame). The diagnostic is
$$\bar I(K) = \lim_{s\ \text{large}} I(Z_A; Z_B \mid K), \qquad \chi(K) = \frac{\partial \bar I(K)}{\partial \log K},$$
where $I$ is mutual information, estimated with a contrastive lower bound. In that estimate, segments from the same utterance are positives and segments from other utterances are negatives. The quantity $\chi$ plays the role of the susceptibility, the rate at which the order parameter responds to the knob. Taking a derivative is the step that the entanglement-derivative work of Qing Xu et al. uses to turn smoothed finite-size curves into sharp peaks. The leakage signal is bounded by the speaker entropy, about $\log 109 \approx 4.7$ nats on VCTK, so a contrastive estimator with batch size 256 can resolve it.
Relation to Prior Work
Three papers motivate the gap. AutoVC shows that a downsampled autoencoder bottleneck alone can strip timbre, but its width must be hand-tuned. AVQVC shows the leakage onset is abrupt in codebook size, but finds it only by grid search. SpeechTokenizer puts semantics in its first quantizer layer only by distilling from HuBERT, and gives no principle for capacity. We do not propose a new converter. We propose a way to locate the threshold these systems tune by hand.
Phase Transitions for the Information Bottleneck in Representation Learning supplies the theoretical expectation that compressed representations change in discrete jumps. It reports 9 transitions on CIFAR-10, matching its conjecture of one fewer than the number of classes. Its threshold condition, however, needs supervised targets and second-order analysis of the loss, which is impractical for speech autoencoders. We replace that condition with an empirical, label-free order parameter.
Learning measurement-induced phase transitions using attention supplies a framing. Its "learnability transition" asks whether two different initial states can be told apart from measurement records alone. Our labeled validation probe asks the same question about two source speakers given only content codes. That paper uses attention to classify an external physical system. We instead apply the information diagnostic to a network's own latent stream.
Measurement-induced criticality in random quantum circuits motivates treating the volume-law entangled phase as an ordered phase in a classical spin model. That is the analogy behind reading leakage as long-range order. We do not claim speech bottlenecks share its universality class or its exact coefficients.
Research Program
The first component is a single training run that sweeps capacity. We take the AVQVC architecture, which has 5.77 million parameters, and give it a nested codebook. In each minibatch only the first $K$ entries are active, with $K$ sampled log-uniformly from 32 to 4096. The decoder is conditioned on $K$. This lets one run produce content codes at every capacity. An ablation checks the shortcut: we train independent models at seven fixed sizes and confirm that nesting does not shift where the transition falls. We repeat the study on AutoVC, with the bottleneck dimension and downsampling factor as the knob, to test whether the phenomenon generalizes beyond vector quantization.
The second component is the diagnostic itself. A small contrastive critic is trained on frozen content codes at each $K$ to estimate $I(Z_A; Z_B)$ as a function of separation $s$. We then read off three things: the plateau $\bar I(K)$, its derivative $\chi(K)$, and the peak location $K_c$.
The estimate needs a confound control. Distant segments can also share information through lexical context, and recording conditions are nearly constant within VCTK. To measure the floor that content alone produces, we compute the same quantity on discretized phone sequences from forced alignment, a purely linguistic stream. VCTK speakers read largely shared prompts, which gives a second control: segment pairs drawn from different speakers reading the same text should show context-driven sharing but not speaker sharing.
The third component treats the number of training speakers $N$ as the "system size." We train on 20, 40, 90 and 109 VCTK speakers, then on LibriSpeech with thousands of speakers. We ask whether $\chi$ curves for different $N$ collapse onto one curve when plotted against $(K - K_c(N))\,N^{a}$ for a fitted exponent $a$. This is exploratory. A clean collapse would let one predict $K_c$ for a new corpus from small pilot runs. Failure to collapse would still leave the label-free detector intact.
Baselines
The naive reference point is AVQVC at its published size of 512, with no tuning. The strongest tuning procedure we compare against is a labeled grid search. At each capacity, we train a speaker classifier on content codes and an ASR model on the converted speech. We then pick the capacity where speaker-identification accuracy on content codes falls to near chance while WER is still near its floor. This is what practitioners would do given labels, so it is the real competitor for locating the threshold. Both procedures get the same models, data splits and compute; the labeled search additionally gets VCTK speaker labels and transcripts. We beat it if our label-free $K_c$ lands within one octave of the labeled threshold on both AVQVC and AutoVC. We would also count it as a clear win if our curve stays as sharp when labels are withheld, for example on LibriSpeech treated as unlabeled.
The strongest existing systems define the bar for the conversion quality reached at $K_c$. AVQVC reports a voice similarity score of 3.29 against 2.91 for AutoVC. W2VC uses WavLM features, an adversarial speaker classifier and phoneme supervision. It reports naturalness MOS 4.45, speaker-similarity MOS 3.62, WER 1.63% against a ground-truth WER of 1.30%, and mel cepstral distortion 8.901. We do not expect to beat W2VC outright with a 5.77M-parameter model, since W2VC uses WavLM features and phoneme labels. The fair claim is narrower. At equal architecture and training budget, the capacity chosen by our diagnostic should match or beat AVQVC's grid-searched 512 on speaker similarity and WER, using one sweep run instead of many.
SpeechTokenizer serves as a reference for teacher-distilled separation. It reports a MUSHRA reconstruction score of 90.55 against 79.86 for EnCodec. We will apply our diagnostic to its first quantizer layer to test whether distillation places that layer on the disordered side of the transition.
Two ablations isolate the key ingredient. In the first, we use the raw plateau $\bar I$ with a fixed threshold instead of the derivative peak, which tests whether the derivative adds sharpness. In the second, we use short-range rather than long-range mutual information, which tests whether long-range sharing specifically is what tracks speaker leakage. We found no reported result on Lacuna for speaker-identification accuracy or EER measured on AVQVC or AutoVC content codes. The labeled probe numbers will therefore be produced in this study.
Why not just…?
Why not just grid-search with a labeled speaker probe? On VCTK this is a real threat. Speaker labels are free there, and a probe directly measures leakage. If the labeled search and our diagnostic agree, the probe needs no improvement on labeled data. Our case then rests on two settings. One is unlabeled or weakly labeled audio, which is where speech tokenizers are trained and where speaker labels do not exist. The other is the single sweep run, which removes the need for repeated training and listening tests. The experiments report both procedures side by side. If our diagnostic is no sharper or cheaper than a probe trained on pseudo-speaker clusters, we will say so.
Why not just watch the reconstruction loss or codebook usage for an elbow? The information-bottleneck transitions paper suggests the loss itself should kink at transitions, so this could work as well as our diagnostic. The doubt is that the decoder already receives a speaker embedding. Leaking timbre into content therefore buys little reconstruction gain, and the kink may be invisible against the content-driven improvement. We test this directly by including loss curvature and codebook perplexity as competing detectors. If either localizes $K_c$ as well as $\chi$, we will adopt the simpler signal and report the long-range diagnostic as an explanation of why the elbow exists.
Why not just add an adversarial speaker classifier and stop worrying about capacity? W2VC shows this route reaches high quality. It also shows that the adversary alone was not enough: W2VC needed phoneme supervision with CTC to keep content intact. The approach therefore assumes the labels our setting lacks. Our diagnostic also applies to adversarial systems. We will measure whether W2VC-style training moves $K_c$ upward, which would show how much capacity the adversary actually reclaims.
Why not just use self-supervised features as the content stream? S2VC and W2VC both report that CPC and WavLM features keep speaker cues. Both needed instance normalization or adversarial purification on top. Running our diagnostic on frozen WavLM and HuBERT layers quantifies that residual leakage without labels, and serves as a second test that $\bar I$ ranks known-leaky representations correctly.
Decisive Experiments
The first decisive check uses the nested-codebook AVQVC on VCTK with the 90/19 speaker split from AVQVC. The $\chi(K)$ peak should fall between 512 and 1024. It should coincide, within one octave, with the point where a held-out speaker classifier on content codes rises above chance. If $\chi$ shows no peak, or its peak is uncorrelated with labeled leakage, the core claim fails.
The second check is the controls. On the phone-sequence stream and on cross-speaker same-text pairs, $\bar I$ should show no transition. This would confirm that the signal is speaker identity and not lexical context.
The third check tests whether optimal conversion sits at the transition. We train fixed-size models densely around $K_c$ and evaluate on unseen-to-unseen CMU ARCTIC conversions. Metrics are speaker similarity (MOS and speaker-verification cosine), WER from an off-the-shelf recognizer, and mel cepstral distortion. The criticality hypothesis predicts that the joint optimum lies within the $\chi$ peak width. A clear optimum well inside the disordered phase would refute it.
The fourth check tests transfer. Repeating the sweep on AutoVC and on unlabeled LibriSpeech, the diagnostic should still produce a sharp peak. On LibriSpeech, the capacity it selects should give a converter whose leakage, verified afterward with held-out speaker labels, is no worse than the VCTK-tuned setting.
References
- Avqvc: One-Shot Voice Conversion By Vector Quantization With Applying Contrastive Learning
- Measurement-induced criticality is tomographically optimal
- AutoVC: Zero-Shot Voice Style Transfer with Only Autoencoder Loss
- SpeechTokenizer: Unified Speech Tokenizer for Speech Language Models
- Phase Transitions for the Information Bottleneck in Representation Learning
- Learning measurement-induced phase transitions using attention
- Measurement-induced criticality in random quantum circuits
- W2VC: WavLM representation based one-shot voice conversion with gradient reversal distillation and CTC supervision
- S2VC: A Framework for Any-to-Any Voice Conversion with Self-Supervised Pretrained Representations