Machine Learning for Antimicrobial Resistance

John Santerre, James J. Davis, Fangfang Xia, Rick StevensarXiv 2016 · DOI · arXiv

Abstract

Biological datasets amenable to applied machine learning are more available today than ever before, yet they lack adequate representation in the Data-for-Good community. Here we present a work in progress case study performing analysis on antimicrobial resistance (AMR) using standard ensemble machine learning techniques and note the successes and pitfalls such work entails. Broadly, applied machine learning (AML) techniques are well suited to AMR, with classification accuracies ranging from mid-90% to low-80% depending on sample size. Additionally, these techniques prove successful at identifying gene regions known to be associated with the AMR phenotype. We believe that the extensive amount of biological data available, the plethora of problems presented, and the global impact of such work merits the consideration of the Data-for-Good community.

Introduction

Antimicrobial resistance (AMR)—the ability of bacteria to survive the drugs meant to kill them—is a mounting global health threat. While large research institutions have the resources to apply machine learning to biology, non-profits and medical facilities in underserved communities often lack the specialized IT talent to bridge the gap between raw genomic data and actionable medical insights. This study serves as a case study demonstrating that "off-the-shelf" machine learning techniques, specifically Random Forest ensembles, can effectively predict AMR from bacterial genomic data with high accuracy. Beyond prediction, these models successfully identify the specific genetic regions responsible for resistance, offering a powerful tool for the "Data-for-Good" community.

Methodology and Datasets

The researchers framed the identification of AMR as a supervised classification task using publicly available data from the Pathosystems Resource Integration Center (PATRIC). The analysis focused on major bacterial species including Streptococcus pneumoniae, Acinetobacter baumannii, Mycobacterium tuberculosis, and Staphylococcus aureus.

Data Preprocessing ($k$-mers)

Raw DNA sequences were broken down into $k$-mers—fixed-length short strands of DNA (typically length 10–50). Each bacterial isolate was represented by a vector of its unique $k$-mer counts. The choice of $k$ is a critical hyperparameter; while small $k$ values capture basic sequence properties, larger $k$ values significantly increase the feature space and improve classification performance.

ROC curves for M. tuberculosis showing improved AUC with larger k-mer sizes.

Algorithms

The primary tool used was the Random Forest (RF) classifier. The study also evaluated AdaBoost, Lasso, and Logistic Regression. By analyzing feature importance within the Random Forest, the authors mapped prioritized $k$-mers back to specific gene regions known to be associated with resistance.

Key Results

The models achieved classification accuracies as high as 92% using standard train/test splits. A significant finding was the relationship between dataset size and model performance.

Performance Plateau

Accuracy generally increased with the number of isolates but tended to plateau on larger datasets. This suggests that even moderately sized genomic databases (a few hundred to a thousand samples) are sufficient for high-quality predictions, making the approach viable even when data is not extremely abundant.

Accuracy plateaus as the number of subsampled isolates increases across different species datasets.

Stability and Biological Validation

One of the most surprising findings was that even with very small sample sizes (as few as 25 samples), the Random Forest could consistently identify top gene regions associated with resistance. These feature importance rankings successfully highlighted gene regions already documented in biological literature, validating the model's predictive "reasoning."

Discussion and Future Work

The primary barrier for machine learning professionals in this field is not algorithmic complexity, but rather the difficulty of formatting biological data and navigating sparse documentation. Furthermore, the current $k$-mer approach creates highly redundant feature sets. Future work should focus on "collapsing" these features into more meaningful biological blocks to improve computational efficiency.

This case study proves that standard classifiers applied to public data can solve real-world biological problems. We urge the machine learning community to look toward public biological datasets as a high-impact area for "Data-for-Good" initiatives.

Summary updated

Abstract

Biological datasets amenable to applied machine learning are more available today than ever before, yet they lack adequate representation in the Data-for-Good community. Here we present a work in progress case study performing analysis on antimicrobial resistance (AMR) using standard ensemble machine learning techniques and note the successes and pitfalls such work entails. Broadly, applied machine learning (AML) techniques are well suited to AMR, with classification accuracies ranging from mid-90% to low- 80% depending on sample size. Additionally, these techniques prove successful at identifying gene regions known to be associated with the AMR phenotype. We believe that the extensive amount of biological data available, the plethora of problems presented, and the global impact of such work merits the consideration of the Data- for-Good community.
BibTeX
@misc{santerre2016machine,
  author = {John Santerre and James J. Davis and Fangfang Xia and Rick Stevens},
  title = {{Machine Learning for Antimicrobial Resistance}},
  year = {2016},
  eprint = {1607.01224},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1607.01224},
}

Loading related content...

Figure