Hamiltonian Monte Carlo with strict convergence criteria reduces run-to-run variability in forensic DNA mixture deconvolution

Mateusz Susik, Holger Schönborn, Ivo F. Sbalzarini

TL;DR

Forensic DNA mixture analysis, crucial for criminal investigations, often suffers from unreliable likelihood ratio (LR) results due to variability in computational methods. This paper introduces a novel Hamiltonian Monte Carlo (HMC) approach that significantly reduces this run-to-run variability by employing stricter convergence criteria, leading to more precise and trustworthy DNA evidence interpretation without sacrificing computational speed.

What Problem Does This Paper Address?

In forensic genetics, analyzing DNA mixtures (samples containing DNA from multiple individuals) is a common but complex task. The strength of evidence in a criminal trial is often quantified by a Likelihood Ratio (LR), which compares the probability of the evidence under two competing hypotheses (e.g., suspect is included vs. suspect is excluded). Probabilistic genotyping (PG) software uses Markov Chain Monte Carlo (MCMC) methods to estimate these LRs. However, a significant problem is the "run-to-run variability" – obtaining substantially different LRs even when running the same analysis on the same data with the same software settings. This uncertainty is problematic because LRs directly influence legal decisions. While this variability was previously attributed to the inherent stochasticity of MCMC, this paper proposes that it's more likely due to the choice of convergence criteria used to determine when the MCMC has run long enough to produce stable results.

Methods

The core of this paper is the development and application of a Hamiltonian Monte Carlo (HMC) algorithm for DNA mixture deconvolution. Unlike traditional MCMC methods that explore the parameter space by taking random steps, HMC uses Hamiltonian dynamics to simulate a physical system, allowing for more efficient exploration of complex probability distributions. A key innovation is the formulation of a probabilistic model that marginalizes out the discrete genotype sets. This creates a purely continuous and differentiable model, which is essential for HMC and allows for the application of strict convergence diagnostics like the Gelman-Rubin (GR) diagnostic. The authors enforce a strict GR threshold (1.05), which is more stringent than typically used in existing PG software. To handle the computational burden of this approach, they also developed a deduplication system that avoids redundant calculations for identical genotype set contributions and implemented GPU acceleration for faster inference.

Datasets

The authors validate their method using publicly available benchmark datasets:

  • MIX05 and MIX13: These are inter-laboratory studies organized by NIST, involving DNA mixtures analyzed with various kits.
  • ProvedIt dataset: This dataset was used to determine the precision of STRmix™.

The paper also mentions using the NIST 1036 U.S. Population Dataset for background allele frequencies.

Key Insights and Ideas

  • Convergence criteria are critical for precision: The paper strongly suggests that the choice of convergence metric and its strictness is a primary driver of run-to-run variability in PG, rather than solely the stochastic nature of MCMC.
  • Marginalizing discrete dimensions simplifies convergence: By reformulating the model to eliminate discrete genotype sets, the authors create a continuous space where standard convergence diagnostics can be applied more effectively and strictly.
  • HMC offers a performance advantage: Despite the increased computational demands of a continuous model and strict convergence, HMC, combined with GPU acceleration and a deduplication system, achieves competitive or even faster runtimes compared to existing methods.
  • Stricter convergence leads to an order of magnitude improvement: The most striking finding is the substantial reduction in LR variability, often by an order of magnitude, when employing the proposed HMC method with strict convergence criteria.

Key Results

  • Reduced Run-to-Run Variability: The HMC method with strict convergence criteria reduced the standard deviation of log10 LR by approximately an order of magnitude compared to STRmix™™™ on benchmark datasets.
  • Competitive Runtimes: Despite the advanced methodology, the HMC implementation achieved fast inference times, with runtimes for 3-contributor mixtures under 7 minutes, 4-contributor mixtures under 35 minutes, and 5-contributor mixtures under an hour on consumer-grade hardware.
  • Accurate LR Reproduction: The algorithm successfully reproduced previously published LR results for benchmark mixtures, validating its accuracy.
  • Improved Precision: The paper demonstrates that stricter convergence criteria, enabled by their continuous model, significantly improve the precision of LR estimates.

Limitations and Future Work

  • Computational Cost of Strict Convergence: While HMC is efficient, achieving very strict convergence (e.g., GR close to 1) can still be computationally intensive, though the authors show it's manageable.
  • Comparison with other Algorithmic Approaches: The authors plan to compare their method with other algorithms using the same probabilistic model and to analyze the trade-offs between Bayesian inference and maximum likelihood estimation.
  • Further Model Improvements: Future work will focus on refining the model and exploring other factors that influence overall precision in forensic DNA analysis, such as population genetics, sample quality, and hyper-parameter settings.
  • Exploration of Worst-Case Precision: The authors suggest investigating the factors that contribute most to the largest influences on final precision and bounding the worst-case precision.

Summary updated

Abstract

We present the use of a Hamiltonian Monte Carlo (HMC) algorithm that reduces run-to-run variability in forensic DNA mixture deconvolution by around an order of magnitude without increased runtime. We achieve this by enforcing strict convergence criteria. We show that the choice of convergence metric strongly influences precision. We validate our method by reproducing previously published results for benchmark DNA mixtures (MIX05, MIX13, and ProvedIt). We also present a complete software implementation of our algorithm that is able to leverage GPU acceleration for the inference process. In the benchmark mixtures, on consumer-grade hardware, the runtime is less than 7 min for 3 contributors, less than 35 min for 4 contributors, and less than an hour for 5 contributors with one known contributor.
BibTeX
@inproceedings{susik2022hamiltonian,
  author = {Mateusz Susik and Holger Schönborn and Ivo F. Sbalzarini},
  title = {{Hamiltonian Monte Carlo with strict convergence criteria reduces run-to-run variability in forensic DNA mixture deconvolution}},
  booktitle = {Forensic},
  year = {2022},
  doi = {10.1016/j.fsigen.2022.102744},
}
Figure