Abstract
Predicting antimicrobial resistance using MALDI-TOF mass spectrometry based machine learning is a fast-growing field of research. Recent advances in machine learning methods specifically designed for MALDI-TOF mass spectra have outperformed established classification approaches. However, classification performance was observed to have a large standard deviation between different train–test splits. We hypothesise that this variance is caused by the underlying phylogenetic structure between microbial samples, which is implicitly reflected in their MALDI-TOF MS profiles, but not taken into account during the training of a model. In this paper, we propose to infer this structure from the dataset—using agglomerative hierarchical clustering—and consider it during the dataset splitting between train and test. We show that incorporating such phylogenetic structure into the antimicrobial resistance prediction scenario leads to an improved classification performance. Average precision was increased from 42.3 to 47.1 for ciprofloxacin resistance prediction in Escherichia coli and from 44.6 to 50.8 for amoxicillin-clavulanic acid resistance prediction in Staphylococcus aureus using a Gaussian process classifier with a MALDI-TOF MS specific kernel.
Introduction
Antimicrobial resistance (AMR) is a critical global health threat. Rapid identification of resistant pathogens is essential for effective patient treatment and stewardship. Matrix-assisted laser desorption/ionization time-of-flight mass spectrometry (MALDI-TOF MS) is already standard in clinical microbiology for species identification. Leveraging this data for AMR prediction could significantly reduce the time-to-result compared to traditional culture-based methods.
However, a significant challenge in training machine learning models for AMR prediction is the high variance in performance across different data splits. This study addresses the hypothesis that this variance stems from the underlying phylogenetic structure of bacterial populations. Standard random splitting often fails to account for the fact that bacteria belong to distinct lineages. If a specific lineage is present in the test set but absent from the training set, the model's ability to generalize is severely compromised. Since genetic sequencing is often unavailable in clinical settings, models have historically been "blind" to this biological diversity.
Methods: Hierarchical Stratification
To mitigate the impact of hidden phylogenetic structure, the authors introduce hierarchical stratification. This approach uses unsupervised learning to uncover the biological diversity within the mass spectrometry data before training.
The process involves four key steps:
- Agglomerative Hierarchical Clustering: Mass spectra are grouped into clusters using linkage methods such as Ward’s or Average linkage. These clusters serve as proxies for the unknown phylogenetic branches of the bacteria.
- Meta-Labeling: Each sample is assigned a "meta-label" that combines its resistance status (Resistant vs. Susceptible) with its assigned cluster ID.
- Stratified Splitting: Data is divided into training and testing sets such that each meta-label is represented proportionally in both. This ensures the model encounters a representative sample of the population's diversity during training.
- Classification: The study evaluates this approach using Logistic Regression and a specialized Gaussian Process classifier with a Peak Information Kernel (GP-PIKE), which is optimized for the sparse nature of MALDI-TOF MS data.
Datasets
The methodology was validated using a subset of the DRIAMS-A database, a comprehensive collection of clinical bacterial isolates. The study focused on three major pathogens: Escherichia coli, Klebsiella pneumoniae, and Staphylococcus aureus.
The dataset includes nine distinct species-antibiotic scenarios, providing a robust testing ground for the proposed stratification method. The number of samples per species ranged from approximately 1,800 to over 3,000, with varying prevalence rates of resistance as detailed in the table above.
Results
Hierarchical stratification led to improved prediction performance in seven out of the nine scenarios tested. Notable improvements were observed in:
- Staphylococcus aureus: Average Precision (AP) for amoxicillin-clavulanic acid resistance increased from 44.6 to 50.8 using the GP-PIKE model.
- Escherichia coli: AP for ciprofloxacin resistance prediction rose from 42.3 to 47.1.
The results confirm that the GP-PIKE model generally outperforms standard Logistic Regression, highlighting the value of domain-specific kernels in processing mass spectrometry data. Furthermore, the study found that even a small number of clusters (typically $k=3$ to $k=6$) was sufficient to capture the essential biological structure needed to enhance model performance.
Discussion and Future Work
While hierarchical stratification improved average performance, it did not significantly reduce the standard deviation between splits. This suggests that while the models are more accurate on average, sample size remains a critical factor for stability, particularly in species with fewer available isolates like K. pneumoniae.
Future research should focus on validating these clusters against ground-truth genomic data to confirm their alignment with biological phylogeny. Additionally, exploring more sophisticated distance metrics, such as Optimal Transport, could further refine the clustering process. Automating the selection of the optimal number of clusters ($k$) will be a vital step toward implementing these models in routine clinical diagnostics.
By incorporating the inherent biological structure of microbial populations into the machine learning pipeline, this work moves us closer to reliable, rapid AMR detection, potentially saving lives and reducing healthcare costs.
Summary updated