Bayesian Strategies for Likelihood Ratio Computation in Forensic Voice Comparison with Automatic Systems

Daniel Ramos, Juan Maroñas, Alicia Lozano-DiezarXiv 2019 · OpenReview

Abstract

This paper explores several strategies for Forensic Voice Comparison (FVC), aimed at improving the performance of the Likelihood Ratios (LRs) when using generative Gaussian score-to-LR models. First, different anchoring strategies are proposed to adapt the LR computation process to the specific case while respecting defined propositions. Second, a fully-Bayesian Gaussian model is employed to address the sparsity in training scores often encountered with these anchoring strategies. Experiments using the NIST 2014 i-Vector challenge set-up demonstrate that the fully-Bayesian model significantly outperforms the standard Maximum-Likelihood approach, providing high robustness in data-sparse scenarios.

Introduction

In forensic voice comparison, experts provide a Likelihood Ratio (LR) to quantify the strength of evidence relative to two competing hypotheses: the prosecution's ($H_p$: the suspect is the speaker) and the defense's ($H_d$: someone else is the speaker). Providing reliable LRs is challenging due to acoustic variability—stemming from transmission channels, background noise, and speaker states—and the inherent sparsity of data in criminal cases. Standard statistical models, such as Maximum Likelihood (ML), often perform poorly with limited data, producing extreme and unreliable LR values that can lead to miscarriages of justice.

Methodology

To address these challenges, this research proposes two main strategies: anchoring and a fully-Bayesian framework.

Anchoring Strategies

Anchoring involves selecting training data that mimics the specific conditions of the forensic case. Two strategies were compared:

  • Suspect-Anchored (SA): Training scores are generated using any available recordings from the suspect.
  • Reference-Anchored (RA): Training scores are specifically anchored to the actual reference recording used in the case, ensuring the model better reflects the specific acoustic conditions of the evidence.

Fully-Bayesian Gaussian Model

Standard systems typically use ML to estimate the mean and variance of score distributions as fixed parameters. In contrast, the fully-Bayesian (BI) approach treats these parameters as random variables and marginalizes over them. This process results in a Student’s t-distribution rather than a Gaussian. The "heavier tails" of the t-distribution naturally account for uncertainty, preventing the model from outputting overconfident, extreme LRs when data is sparse.

Results and Discussion

The study utilized the NIST 2014 i-Vector Challenge dataset, involving over 12 million trials to test robustness.

LLR Moderation

A critical finding is the "self-moderating" nature of the Bayesian approach. As illustrated in the comparison of transformation functions, the Bayesian log-likelihood ratio (LLR) remains significantly more conservative at the extremes of the score distribution compared to the ML-based LLR, which tends toward infinity.

The Bayesian transformation function (red) produces much more moderate LLR values at the score extremes than the ML function (blue).

Robustness to Data Sparsity

The performance was measured using $C_{llr}$ (cost of the log-likelihood ratio), where lower values indicate better calibration. The Bayesian model's advantage is most pronounced when the number of training scores is very low. In the Suspect-Anchored scheme, the ML model's performance degrades rapidly as data decreases, reaching a $C_{llr}$ near 12 with only two training scores. Conversely, the Bayesian model remains stable and informative, with $C_{llr}$ values consistently below 1.0 even with a single training score.

Performance comparison for the Suspect-Anchored scheme. The Bayesian model (blue) is significantly more robust than ML (red) as the number of training scores decreases.

Conclusion

The integration of a fully-Bayesian framework with Reference-Anchored data selection provides a robust solution for forensic voice comparison. By accounting for parameter uncertainty through predictive t-distributions, the system avoids the dangerous overconfidence of traditional ML methods in small-data regimes, ensuring more reliable evidence for judicial proceedings.

Summary updated

Abstract

This paper explores several strategies for Forensic Voice Comparison (FVC), aimed at improving the performance of the LRs when using generative Gaussian score-to-LR models. First, different anchoring strategies are proposed, with the objective of adapting the LR computation process to the case at hand, always respecting the propositions defined for the particular case. Second, a fully-Bayesian Gaussian model is used to tackle the sparsity in the training scores that is often present when the proposed anchoring strategies are used. Experiments are performed using the 2014 i-Vector challenge set-up, which presents high variability in a telephone speech context. The results show that the proposed fully-Bayesian model clearly outperforms a more common Maximum-Likelihood approach, leading to high robustness when the scores to train the model become sparse.
BibTeX
@misc{ramos2019bayesian,
  author = {Daniel Ramos and Juan Maroñas and Alicia Lozano-Diez},
  title = {{Bayesian Strategies for Likelihood Ratio Computation in Forensic Voice Comparison with Automatic Systems}},
  year = {2019},
  eprint = {1909.08315},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1909.08315},
}
Figure