Evaluating the dose, indication and agreement with guidelines of antimicrobial use in companion animal practice with natural language processing

Brian Hur, Laura Y. Hardefeldt, Karin Verspoor, Timothy Baldwin, James R. GilkersonJAC-Antimicrobial 2021 · DOI

TL;DR

This paper leverages natural language processing (NLP) to analyze millions of veterinary clinical notes, revealing that while antimicrobial use is common for skin disorders and traumatic injuries in pets, a significant portion of prescriptions do not align with established guidelines, particularly concerning dosage. This work highlights the potential of NLP for large-scale antimicrobial stewardship monitoring in veterinary medicine, while underscoring the critical need for more complete and standardized clinical record-keeping.

What Problem Does This Paper Address?

Antimicrobial resistance (AMR) is a growing global health crisis, and veterinary practices play a role in its emergence. Antimicrobial stewardship programs are crucial for mitigating AMR by ensuring appropriate antimicrobial use. A key aspect of this is selecting the right drug, at the right dose, for the right duration, and for the correct indication. However, extracting this detailed information from veterinary clinical records has been a significant challenge. Traditional methods often rely on structured data fields or insurance claims, which may not capture the full clinical context. This paper aims to bridge this gap by using NLP to automatically extract and analyze antimicrobial usage patterns, including dose and indication, directly from unstructured veterinary clinical notes, thereby enabling a more comprehensive evaluation of antimicrobial stewardship.

Methods

The researchers employed natural language processing (NLP) techniques, specifically a pre-trained contextualized language model called "VetBERT," to analyze veterinary clinical records. VetBERT was fine-tuned on a large corpus of veterinary notes from the VetCompass Australia database to accurately classify the indication for antimicrobial use. The NLP algorithms were designed to extract not only the diagnosis but also the dose, duration of therapy, and specific antimicrobial agents administered. To validate the accuracy of the NLP-extracted dosage calculations, a manual calculation was performed on a subset of records, achieving a 94% match. The model's accuracy in identifying indications was also assessed, demonstrating strong performance.

Datasets

The study utilized de-identified clinical data from 137 companion animal practices in Australia, sourced from the VetCompass Australia database (Version 0.3). This dataset comprised over 4.4 million consultation records from approximately 513,962 dogs and 199,358 cats. The data spanned from January 1, 2013, to December 31, 2017. No new benchmarks were introduced; rather, the study leveraged an existing large-scale veterinary data repository.

Key Insights and Ideas

  • NLP as a powerful tool for veterinary data analysis: The study demonstrates that NLP can effectively extract complex clinical information from unstructured text, opening doors for large-scale antimicrobial stewardship monitoring in veterinary medicine.
  • Incomplete records are a major hurdle: Despite the power of NLP, the study highlights that the completeness of clinical records is paramount. A significant portion of records lacked essential information (dose, duration, indication), limiting the ability to fully assess antimicrobial appropriateness.
  • Dosing inconsistencies are prevalent: A key finding is that a substantial percentage of antimicrobial doses administered to companion animals were not consistent with guideline recommendations, suggesting a potential area for targeted intervention.
  • Common indications and drug choices: Skin disorders and traumatic injuries emerged as the most frequent reasons for antimicrobial prescriptions in dogs and cats, respectively. The choice of antimicrobials for these conditions often aligned with guidelines, but variations were noted.

Key Results

  • Out of over 4.4 million consultation records, systemic antimicrobials were administered in 109,719 cat consultations (9.7%) and 343,668 dog consultations (11%).
  • Complete data for indication, dose, and duration were extracted for only 38% of consultations involving antimicrobial use.
  • When complete data were available, the dose was consistent with guideline recommendations in 73% of cases.
  • Skin disorders were the most common indication in dogs (19% of total antimicrobials), while traumatic injuries were most common in cats (15% of total antimicrobials).
  • Doxycycline showed the lowest proportion of guideline-compliant doses in dogs (35%), and enrofloxacin in cats (25%).

Limitations and Future Work

The authors acknowledge several limitations:

  • Incomplete Data: The primary limitation is the incompleteness of clinical records, with only 38% of records containing all necessary information for a full assessment of antimicrobial appropriateness. This significantly impacts the scope of analysis.
  • Syndrome-Level Diagnosis: The NLP extracted diseases at a syndrome level, lacking the detailed clinical specifics needed for a nuanced evaluation of appropriateness.
  • Repeat Visits and Chronic Conditions: Evaluating the duration of therapy for chronic conditions or repeat visits proved challenging due to the study's methodology.
  • Lack of Culture and Susceptibility Data: The absence of culture and susceptibility testing results in the analyzed data limits the ability to assess the selection of antimicrobials based on specific pathogen sensitivities.

Future work suggested includes:

  • Further research into the reasons for missing data in clinical records.
  • Developing methods to incorporate clinical pathology data (like culture and susceptibility testing) into the analysis.
  • Exploring more detailed disease syndrome analysis.
  • Investigating the impact of factors like patient weight variability and label discrepancies on dosing.
  • Evaluating the utility of NLP techniques in general medical practice for similar stewardship evaluations.

Summary updated

Abstract

Automated extraction using NLP methods is a powerful tool to evaluate large datasets and to enable veterinarians to describe the reasons that antimicrobials are administered. However, this can only be determined when the data presented in the clinical record are complete, which was not the case in most instances in this dataset. Most importantly, the dose administered varied and was often not consistent with guideline recommendations.
BibTeX
@inproceedings{hur2021evaluating,
  author = {Brian Hur and Laura Y. Hardefeldt and Karin Verspoor and Timothy Baldwin and James R. Gilkerson},
  title = {{Evaluating the dose, indication and agreement with guidelines of antimicrobial use in companion animal practice with natural language processing}},
  booktitle = {JAC-Antimicrobial},
  year = {2021},
  doi = {10.1093/jacamr/dlab194},
}
Figure