Revisiting and Improving Scoring Fusion for Spoofing-aware Speaker Verification Using Compositional Data Analysis

Xin Wang, Tomi Kinnunen, Kong Aik Lee, Paul-Gauthier Noé, Junichi Yamagishi

TL;DR

This paper provides a rigorous decision-theoretic foundation for score-level fusion in Spoofing-aware Speaker Verification (SASV) systems. By framing SASV through the lens of compositional data analysis, the authors demonstrate that while simple score summation can be mathematically justified, it requires strict score calibration to prevent one subsystem from dominating. Furthermore, they reveal that linear fusion is fundamentally sub-optimal, proposing a novel non-linear fusion method that significantly outperforms traditional heuristics on the SASV challenge benchmark.

What Problem Does This Paper Address?

Automatic Speaker Verification (ASV) systems are designed to confirm a speaker's identity, but they are highly vulnerable to voice spoofing attacks (such as deepfakes or voice cloning). To counter this, researchers combine ASV with Spoofing Countermeasures (CM). An integrated Spoofing-aware Speaker Verification (SASV) system must handle three distinct scenarios: accepting a true speaker's real voice (target bona fide), rejecting an impostor's real voice (non-target bona fide), and rejecting any synthesized/spoofed voice (spoof).

A common and intuitive way to build an SASV system is score-level fusion, where the individual scores from an ASV system ($s_{\text{asv}}$) and a CM system ($s_{\text{cm}}$) are combined—often by simply adding them together ($s_{\text{sasv}} = s_{\text{asv}} + s_{\text{cm}}$)—to make a final accept/reject decision.

Figure 1: Example SASV with score-level fusion (B1 in [5])

While intuitively reasonable, these summation approaches have largely been heuristic. Without a solid theoretical framework, the implicit assumptions behind score summation remain unclear, leaving substantial room for sub-optimal decision-making.

Methods

The authors tackle this problem by reformulating SASV as a ternary classification task (three classes) that requires a binary action (accept or reject). They analyze this setup using two main frameworks:

  1. Compositional Data Analysis (CoDA): The authors use an isometric log-ratio (ILR) transformation to map class posterior probabilities into a 2D Euclidean space. This mathematical framework reveals that summing ASV and CM scores is actually a theoretically sound way to weigh evidence, but only if the scores strictly approximate Log-Likelihood Ratios (LLRs). This underscores the absolute necessity of score calibration before fusion.
  2. Bayesian Decision Theory: By analyzing the system from the perspective of minimizing overall decision cost, the authors prove that a linear combination (summation) of LLRs cannot produce an optimal decision boundary. Instead, they derive an optimal non-linear fusion method based on a log-sum-exp operation of the calibrated LLRs:

$$s_{\text{sasv}} = -\log \left[ (1 - \rho)e^{-\text{llr}_{\text{non.bf}}^{\text{tar.bf}}(\mathbf{s})} + \rho e^{-\text{llr}_{\text{spf}}^{\text{tar.bf}}(\mathbf{s})} \right]$$

Where $\rho$ represents the spoof prevalence prior.

Datasets

The methods were evaluated using the official training, development, and evaluation splits of the SASV Challenge 2022 database.

  • ASV Component: Pre-trained ECAPA-TDNN.
  • CM Component: AASIST-based embedding extractor.

Key Insights and Ideas

  • The Danger of Raw Score Summation: In raw heuristic fusion, the dynamic ranges of ASV and CM scores are often wildly different. As a result, one score completely dominates the decision, rendering the other subsystem effectively useless.
  • Calibration is the Equalizer: Applying a simple affine transformation ($f(x) = ax + b$) to calibrate raw scores into proper LLRs before summing them completely changes the game, properly balancing the influence of both subsystems.
  • Linear Boundaries are Sub-optimal: Even when using perfect LLRs, a linear decision boundary (a straight line in the score space) will always falsely accept or reject certain marginal samples. The optimal decision boundary is inherently curved (non-linear), bending to safely isolate the target bona fide samples from both spoofing attacks and random impostors.
Scatter plot showing that the optimal non-linear decision boundary (solid lines) safely isolates target samples compared to the sub-optimal linear boundary (dashed line).

Key Results

The paper compares the baseline heuristic addition against calibrated linear and non-linear fusion methods using the SASV Equal Error Rate (SASV-EER):

  • Baseline (B1 - Raw Summation): Achieved a poor SASV-EER of 20.46% because the CM score dominated the fused score.
  • Calibrated Summation (B1c): Simply calibrating the scores before adding them caused the SASV-EER to plummet to 2.73%.
  • Generative LLR Fusion (L2c - Linear): Treating scores as features and applying generative calibration further reduced the EER to 1.56%.
  • Optimal Non-linear Fusion (L3c): Implementing the proposed non-linear log-sum-exp fusion achieved the best overall performance with an SASV-EER of 1.43%.

Limitations and Future Work

The primary limitation of the proposed optimal non-linear fusion method is that the prior-dependent parameter $\rho$ (the spoof prevalence prior) is tightly coupled with the LLRs. It cannot be mathematically decoupled, meaning system designers must estimate it empirically via grid search on a development set.

For future work, the authors expect this framework to foster deeper studies into how different application requirements (e.g., high-security banking vs. low-friction smart-home devices) alter the optimal non-linear boundary by adjusting specific false-acceptance and false-rejection costs.

Summary updated

Abstract

Fusing outputs from automatic speaker verification (ASV) and spoofing countermeasure (CM) is expected to make an integrated system robust to zero-effort imposters and synthesized spoofing attacks. Many score-level fusion methods have been proposed, but many remain heuristic. This paper revisits score-level fusion using tools from decision theory and presents three main findings. First, fusion by summing the ASV and CM scores can be interpreted on the basis of compositional data analysis, and score calibration before fusion is essential. Second, the interpretation leads to an improved fusion method that linearly combines the log-likelihood ratios of ASV and CM. However, as the third finding reveals, this linear combination is inferior to a non-linear one in making optimal decisions. The outcomes of these findings, namely, the score calibration before fusion, improved linear fusion, and better non-linear fusion, were found to be effective on the SASV challenge database.
BibTeX
@inproceedings{wang2024revisiting,
  author = {Xin Wang and Tomi Kinnunen and Kong Aik Lee and Paul-Gauthier Noé and Junichi Yamagishi},
  title = {{Revisiting and Improving Scoring Fusion for Spoofing-aware Speaker Verification Using Compositional Data Analysis}},
  booktitle = {Interspeech},
  year = {2024},
  doi = {10.21437/interspeech.2024-422},
}
Figure