Cross-Cycle Harmonic Coherence of the Glottal Residual for Spoof Detection

Spoofing countermeasures try to tell real human speech from speech produced by text-to-speech, voice conversion, or replay through a loudspeaker. One family of them looks at the vocal-fold excitation rather than the familiar spectral envelope. Two signal-processing traditions have each found useful cues here, but separately. The first strips off the vocal tract with linear prediction and studies what is left, a signal called the excitation residual. The second cuts speech into windows aligned to individual pitch cycles, so that phase becomes stable and comparable from one cycle to the next. Nobody has combined them: residual features are always computed with fixed 25 ms windows that slice glottal pulses at arbitrary points. Pitch-synchronous phase features are always computed on the raw waveform, where vocal-tract resonances contaminate the phase.

We propose a representation that brings the two together. It borrows two design rules from neural solvers for time-dependent density functional theory, the quantum-chemistry method that simulates how electrons respond to an applied field. The first rule is to factor out trivial phase rotation analytically before learning anything. The second is to summarize many interacting components with a second-order "density matrix" rather than by pooling them one at a time. The resulting object is a small matrix that measures how consistently the harmonics of the glottal residual keep their phase relationships across consecutive pitch cycles. By construction it does not change when the analysis window is misplaced, and its normalized off-diagonal entries do not change under any fixed linear recording channel. Both properties target the two failures the literature documents: dependence on fragile glottal-instant detection, and collapse on unseen attack types.

Core Claim

Take each pitch cycle $m$ of the linear-prediction residual and compute its complex harmonic amplitudes $h_{m,k}$ at multiples $k = 1,\dots,K$ of that cycle's fundamental frequency. Rotate each harmonic by $k$ times the fundamental's phase $\phi_{m,1}=\arg h_{m,1}$, which gives $\tilde h_{m,k} = h_{m,k}\,e^{-ik\phi_{m,1}}$. Then summarize a block of $M$ cycles by

$$R = \frac{1}{M}\sum_{m=1}^{M} \tilde h_m \tilde h_m^{\dagger}, \qquad C_{kl} = \frac{|R_{kl}|}{\sqrt{R_{kk}R_{ll}}},$$

where $\tilde h_m$ is the $K$-vector of rotated harmonics, $\dagger$ is conjugate transpose, and $C_{kl}\in[0,1]$ is the cross-cycle coherence between harmonics $k$ and $l$.

The two invariances follow from this construction. A timing offset $\tau$ of the cycle window multiplies $h_{m,k}$ by $e^{-ik\omega_0\tau}$ and shifts $\phi_{m,1}$ by $-\omega_0\tau$, so $\tilde h_m$ is unchanged. This means errors in placing the glottal closure instant cancel exactly, to the extent the harmonic model and the $f_0$ estimate hold. A fixed linear channel with response $G$ multiplies each $\tilde h_{m,k}$ by a cycle-independent constant when $f_0$ is locally steady, so $C$ is unchanged.

Our claim is that $C$ isolates something neither tradition measures. That quantity is the cycle-to-cycle regularity of the glottal phase structure. Natural vocal folds produce it with characteristic partial coherence. Vocoders either over-regularize it, which drives coherence toward 1, or scramble it. Loudspeaker nonlinearity and reverberation interacting with changing pitch disturb it. The diagonal of $R$, the per-harmonic energy, keeps the channel-dependent information that replay detection also needs, so the two parts of $R$ can be studied separately.

Where the Pieces Come From

The motivating evidence comes from two groups of papers. On the residual side, New Acoustic Features for Synthetic and Replay Spoofing Attack Detection computes cepstra on the residual and reports a 57.2% relative error reduction over plain linear-frequency cepstra on the ASVspoof 2019 logical-access development set, but only 24.8% on evaluation. Novel Linear Frequency Residual Cepstral Features for Replay Attack Detection maps the residual spectrum through linear filterbanks. Linear Prediction Residual based Short-term Cepstral Features for Replay Attacks Detection uses the residual's Hilbert envelope and finds it far better than a plain residual spectrum. All three use asynchronous frames, so none of them sees cycle-level phase.

On the pitch-synchronous side, Relative phase information for detecting human speech and spoofed speech anchors windows to waveform peaks and normalizes phase by a base frequency. This is exactly the per-cycle rotation we reuse. Its error, however, rises from 0.013% on known attacks to an average 3.726% on evaluation including unseen attacks. Replay attack detection with auditory filter-based relative phase features pools such phase into Mel and Gammatone bands for replay. Both papers keep per-frame phase vectors from the raw waveform, so they never deconvolve the tract and never model how phase relationships persist across cycles.

The source-tract separation work of Doval et al. (source-tract separation) constrains the design. It shows that exact factorizations break when windows drift from true glottal closure. We therefore deliberately avoid such factorizations and build in offset invariance instead. Maximum a posteriori Speech Enhancement Based on Double Spectrum supplies supporting evidence that statistics across pitch-synchronous frames separate speech from noise far better than asynchronous modulation spectra: it reports a speech–noise divergence of 3.86 against 0.52. It works on the full waveform and on magnitudes, not on phase coherence of the residual.

The two design rules come from Orbital Transformers for Predicting Wavefunctions in Time-Dependent Density Functional Theory. That paper factors out the known phase evolution of the unperturbed state (a "delta transformation") and finds that message passing through a contracted density matrix beats pooling individual orbitals. We make no physical claim linking electrons and vocal folds. We take only these two rules, and the ablations below test each of them directly.

Research Program

We will build the pipeline in four stages. First, we run per-frame linear-prediction inverse filtering to obtain the residual. Second, we place glottal closure instants and segment the residual into individual cycles. Third, we compute harmonic amplitudes per cycle using that cycle's own period, then apply the phase rotation. Fourth, we compute $R$ over sliding blocks of $M$ cycles; $M$ is swept from 4 to 32.

We will use $C$ in two forms. The compact form is a descriptor: band-averaged off-diagonal coherence plus the purity $\mathrm{Tr}(R^2)/\mathrm{Tr}(R)^2$. Purity is 1 when all $M$ cycles share identical relative phase structure. The full form is the upper triangle of $C$, fed to a back-end classifier.

We will also add a second "delta" variant that computes $R$ on cycle innovations $\tilde h_m - \tilde h_{m-1}$. This variant tests whether modeling only the deviation from the previous cycle helps further.

Unvoiced frames carry no cycle structure. They fall back to the existing residual cepstral features, and the system reports what fraction of each utterance is voiced. That way gains can be attributed honestly.

We will test the analysis claims directly before any classification. Using clean speech with controlled degradations, we will check four things. We will apply synthetic linear filters and confirm that $C$ is unchanged while the diagonal changes. We will inject window jitter and confirm that $C$ is stable. We will measure how coherence shifts under re-synthesis through a vocoder. We will measure how it shifts under simulated loudspeaker nonlinearity and reverberation.

Baselines

The naive reference is magnitude cepstra with a Gaussian mixture back-end, with no excitation or phase processing. On ASVspoof 2015 the file reports 1.74% development EER for MFCC (Relative phase information for detecting human speech and spoofed speech). On ASVspoof 2017 the file reports 21.61% evaluation EER for CQCC (Replay attack detection with auditory filter-based relative phase features). We will reproduce these on our own pipeline rather than trusting the cross-paper numbers.

The strongest phase competitor is pseudo-pitch relative phase. It is the closest existing method, and it already contains the per-cycle phase rotation. On ASVspoof 2015 the combined MFCC, group-delay and relative-phase system averages 3.726% evaluation EER including unseen attacks. On ASVspoof 2017 the best fusion of CQCC with Gammatone relative phase reaches 9.48% evaluation EER, and Mel relative phase alone reaches 10.98%.

The strongest residual competitor is the fused residual-cepstral system on ASVspoof 2019 logical access. It reaches 2.57% evaluation EER (New Acoustic Features for Synthetic and Replay Spoofing Attack Detection).

All baselines and our features will share the same rules. They use the same voice-activity and train/development/evaluation splits. Each feature family is run with both a Gaussian mixture back-end and a light convolutional network, so feature gains are not confounded with classifier gains. Every system gets an equal hyperparameter search budget tuned only on development data. Fusion is equal-weight score fusion with weights fixed on development.

Beating the competitors has a specific meaning here. Standalone, the coherence features must beat standalone relative phase on the ASVspoof 2015 unseen-attack subset. In fusion, they must improve on 3.726% average evaluation EER (2015), 9.48% (2017) and 2.57% (2019 logical access) when added to each paper's own fusion.

The key ablations each change one ingredient. The first applies relative-phase features to the residual instead of the waveform, without the coherence matrix; this is the straightforward reading of the gap. The second mean-pools the rotated per-cycle vectors instead of forming $R$; this isolates the density-matrix rule. The third computes $C$ on the raw waveform instead of the residual. The fourth removes the phase rotation; this isolates the delta rule. The fifth compares glottal-instant detection against peak-picking against deliberately jittered anchors.

Why not just…?

Why not just compute relative phase on the residual and stop there? This is the obvious move, and it is our first ablation. Per-cycle phase vectors describe what each cycle looks like. They do not describe how stable those relationships are across cycles, and that stability is where vocoders and loudspeakers plausibly differ from vocal folds. If residual relative phase matches the coherence features on unseen attacks, the matrix adds nothing, and we will report the simpler feature as the contribution.

Why not just train a deep network on spectrograms or raw waveforms and let it learn these cues? This is a real threat. The file shows that a convolutional network on residual cepstra alone reached 29.81% on 2017 evaluation, while a residual-plus-LFCC network fusion reached 15.00% (Novel Linear Frequency Residual Cepstral Features for Replay Attack Detection). Capacity alone has not closed the gap, but there is no reported result found on Lacuna for raw-waveform end-to-end models on these benchmarks. We will therefore train such a network under the same compute and tuning budget. We will also check whether adding $C$ as an input channel still helps it, because complementary gain over a learned front-end is the question that matters.

Why not just use a better glottal-instant detector with the existing phase features? Better detectors still fail in noise, in breathy voice and at high pitch. The file notes that replay cues behave differently for female speakers (Linear Prediction Residual based Short-term Cepstral Features for Replay Attacks Detection). Our representation is exactly invariant to a constant per-cycle offset. The jitter experiment will show whether EER stays flat as anchor error grows while relative-phase features degrade. If both degrade equally, the invariance argument fails in practice.

Why involve density functional theory at all? It contributes only two design choices, and each has a dedicated ablation. If neither the phase-rotation step nor the second-order matrix helps, the borrowing was unjustified and we will say so.

Proposed Experiments

We will evaluate on ASVspoof 2015 to test generalization to unseen synthesis, reporting the S10 and other unknown-attack subsets separately. ASVspoof 2017 tests replay. ASVspoof 2019 tests both logical and physical access with disjoint speakers. TIMIT with controlled degradations serves the analysis experiments. All results are reported as EER, split by gender and by voiced fraction.

The first decisive check is channel invariance. On 2019 logical access, we train on one condition, then evaluate after passing the evaluation audio through unseen linear filters. Normalized coherence should lose almost nothing, while residual cepstra should degrade.

The second check is anchor robustness. We sweep the window jitter as a fraction of the pitch period and plot EER for our features against pseudo-pitch relative phase.

The third check is the unseen-attack gap. Relative phase's rise from 0.013% to 3.726% is the target. We ask whether coherence features narrow the ratio between development and evaluation error, not only the absolute evaluation EER.

The fourth check addresses the dispute in the file. Tak et al. argue that replay artifacts are spread evenly across frequency, while Oo et al. argue they concentrate in low auditory bands. We will compute per-harmonic-pair contributions to $C$ on 2017 replay data, which gives a direct answer as to where coherence breaks under replay.

If the channel-invariance and jitter checks pass but there is no evaluation gain, the representation is still a principled analysis tool. If all checks fail, the idea is wrong.