Use of machine-learning algorithms to aid in the early detection of leptospirosis in dogs

Krystle L. Reagan, Shaofeng Deng, Junda Sheng, Jamie Sebastian, Zhe Wang, Sara N. Huebner, Louise A. Wenke, Sarah R. Michalak, Thomas Strohmer, Jane E. SykesJournal of Veterinary Diagnostic Investigation 2022 · DOI

TL;DR

This paper introduces machine learning models that can accurately predict leptospirosis in dogs using readily available clinical data from the first day of hospitalization. These models significantly outperform traditional diagnostic methods, offering a faster and more sensitive way to identify this potentially life-threatening zoonotic disease.

What Problem Does This Paper Address?

Leptospirosis is a serious bacterial infection that can cause severe illness in dogs, including kidney and liver damage, and even death. Early and accurate diagnosis is crucial for effective treatment and a good prognosis, as 90% of infected dogs can recover with prompt care. However, current diagnostic methods, particularly the microscopic agglutination test (MAT), are often slow and lack sensitivity in the early stages of the disease. This delay can lead to missed diagnoses, increased costs, and potentially euthanasia, even when the disease is treatable. The authors aim to bridge this diagnostic gap by developing machine learning models that can predict leptospirosis using information available at the initial veterinary visit.

Methods

The researchers developed two machine learning models:

  • Blood Work (BW) Model: This model utilizes signalment (breed, age, sex, weight) and clinicopathologic data (complete blood count, serum chemistry panel, and urinalysis) collected on the first day of hospitalization.
  • BW + MAT Model: This model incorporates the same data as the BW model, plus the initial MAT titer obtained at patient intake.

Both models were trained using a Support Vector Machine (SVM) algorithm with a radial basis function kernel. The models were trained and validated using a repeated 10-fold cross-validation process to tune hyperparameters. For the BW+MAT model, a decision tree was used to stratify patients based on their initial MAT titer before applying a nonlinear SVM.

Datasets

The study utilized retrospective data from the University of California-Davis Veterinary Medical Teaching Hospital.

  • Training Set: Data from dogs with MAT tests performed between January 1, 2000, and December 31, 2018. This included 91 leptospirosis-positive and 322 leptospirosis-negative dogs after exclusions.
  • Test Set: Data from dogs with MAT tests performed between January 1, 2019, and December 31, 2020. This included 9 leptospirosis-positive and 44 leptospirosis-negative dogs after exclusions.

The paper does not introduce a new benchmark dataset but rather uses a retrospective clinical dataset.

Key Insights and Ideas

  • Clinical Data is Highly Predictive: The study demonstrates that routine clinical data, often collected at the first point of contact with a veterinarian, is remarkably effective in predicting leptospirosis. This highlights the value of comprehensive data collection and analysis in veterinary medicine.
  • MAT Limitations: The research reinforces the known limitations of the MAT test, particularly its low sensitivity in the early stages of infection. This underscores the need for complementary diagnostic tools.
  • MLMs Outperform Traditional Methods: The developed machine learning models significantly outperform the initial MAT titer in terms of sensitivity and accuracy, offering a more reliable early screening tool.
  • Minimal Gain from Early MAT: While the BW+MAT model showed a slight improvement in specificity over the BW-only model, the overall performance gain was not substantial, suggesting that the BW model alone is a powerful diagnostic aid.

Key Results

  • High Sensitivity: Both the BW and BW+MAT models achieved 100% sensitivity in predicting leptospirosis in the test set.
  • High Specificity: The BW model achieved 90.9% specificity, while the BW+MAT model achieved 93.2% specificity.
  • Strong AUC: The Area Under the Curve (AUC) for the BW model was 0.955, and for the BW+MAT model was 0.959, indicating excellent discriminatory power.
  • Initial MAT Performance: In contrast, using an initial MAT titer of ≥1:3,200 as a predictor yielded only 42% sensitivity and an AUC of 0.775.

Limitations and Future Work

  • Retrospective Study: The study is retrospective, meaning the data was collected after the fact. This can introduce biases, and the authors acknowledge that their reliance on owner-reported vaccination history might have led to some misclassification.
  • Referral Population: The study population consisted of dogs from a referral hospital, which may not be representative of the general dog population. Dogs presenting to a referral center might have more advanced or complex disease presentations.
  • Limited Geographic Scope: All data was collected from a single geographic location and a single clinical laboratory, which may limit the generalizability of the models to other regions or laboratories with different instrumentation and reagent lots.
  • Future Work: The authors suggest future studies should validate these models in a prospective, multi-institutional setting with larger sample sizes and data from various clinical laboratories. They also propose incorporating data from primary care settings to assess the models' ability to identify earlier or less severe cases of leptospirosis. Further research could also explore the transferability of these models to different geographic regions and laboratory settings.

Summary updated

Abstract

Leptospirosis is a life-threatening, zoonotic disease with various clinical presentations, including renal injury, hepatic injury, pancreatitis, and pulmonary hemorrhage. With prompt recognition of the disease and treatment, 90% of infected dogs have a positive outcome. Therefore, rapid, early diagnosis of leptospirosis is crucial. Testing for <i>Leptospira</i>-specific serum antibodies using the microscopic agglutination test (MAT) lacks sensitivity early in the disease process, and diagnosis can take >2 wk because of the need to demonstrate a rise in titer. We applied machine-learning algorithms to clinical variables from the first day of hospitalization to create machine-learning prediction models (MLMs). The models incorporated patient signalment, clinicopathologic data (CBC, serum chemistry profile, and urinalysis = blood work [BW] model), with or without a MAT titer obtained at patient intake (=BW + MAT model). The models were trained with data from 91 dogs with confirmed leptospirosis and 322 dogs without leptospirosis. Once trained, the models were tested with a cohort of dogs not included in the model training (9 leptospirosis-positive and 44 leptospirosis-negative dogs), and performance was assessed. Both models predicted leptospirosis in the test set with 100% sensitivity (95% CI: 70.1-100%). Specificity was 90.9% (95% CI: 78.8-96.4%) and 93.2% (95% CI: 81.8-97.7%) for the BW and BW + MAT models, respectively. Our MLMs outperformed traditional acute serologic screening and can provide accurate early screening for the probable diagnosis of leptospirosis in dogs.
BibTeX
@article{reagan2022use,
  author = {Krystle L. Reagan and Shaofeng Deng and Junda Sheng and Jamie Sebastian and Zhe Wang and Sara N. Huebner and Louise A. Wenke and Sarah R. Michalak and Thomas Strohmer and Jane E. Sykes},
  title = {{Use of machine-learning algorithms to aid in the early detection of leptospirosis in dogs}},
  journal = {Journal of Veterinary Diagnostic Investigation},
  year = {2022},
  doi = {10.1177/10406387221096781},
}
Figure