Introduction
In human medicine, standardized coding systems like ICD-10 are essential for billing, clinical research, and public health monitoring. Veterinary medicine, however, lacks a similarly robust and automated infrastructure. Most veterinary records are captured as unstructured "SOAP" notes (Subjective, Objective, Assessment, Plan), creating a significant bottleneck for large-scale data analysis. This lack of structured data hinders the identification of patient cohorts for clinical trials and limits cross-species translational research, where pet diseases can offer insights into human health.
DeepTag is a deep learning framework designed to address this gap by automatically assigning standardized SNOMED-CT diagnostic codes to free-text veterinary clinical notes. By combining a hierarchical training objective with a novel "abstention" mechanism, DeepTag bridges the gap between messy clinical narratives and structured medical data.
Methods
DeepTag treats diagnostic coding as a multi-label classification problem using a Bidirectional Long Short-Term Memory (BLSTM) network. This architecture is particularly effective at capturing context within the long, abbreviation-heavy sequences typical of clinical prose.
A key innovation of DeepTag is its Hierarchical Objective. Rather than treating disease codes as independent categories, the model leverages the semantic relationships within the SNOMED-CT hierarchy. By grouping codes into "meta-diseases" and using an $L_2$-based distance objective, the model is forced to place related disease embeddings closer together in vector space. This use of medical hierarchy acts as a form of regularization, improving performance on rarer diseases.
Furthermore, the framework includes a "DeepTag-abstain" layer. This module estimates the model's uncertainty for each note. If the confidence score falls below a certain threshold, the system "abstains" from making a prediction and flags the record for human review, facilitating a high-accuracy hybrid workflow.
Datasets
The model was evaluated using two distinct datasets:
- CSU Dataset: 112,558 veterinary notes from Colorado State University, manually annotated with SNOMED-CT codes. This served as the primary training and internal testing set.
- PP (Private Practice) Dataset: 586 notes from a private practice in California. This "out-of-domain" set tested the model's ability to generalize across different writing styles and disease distributions.
Results and Discussion
DeepTag demonstrated superior performance over standard baselines, including CNNs and MetaMap. On the CSU test set, it achieved a weighted $F_1$ score of 0.824. While performance dropped on the out-of-domain PP dataset ($F_1$ of 0.634), it remained significantly higher than non-hierarchical models, highlighting the importance of the hierarchical training objective.
The abstention mechanism proved highly effective. As shown in the results, deferring the most uncertain cases to human experts leads to a sharp increase in the accuracy of the remaining automated classifications. By abstaining from just 10% of the most difficult cases, the model's performance on the rest of the dataset improved significantly, outperforming simple confidence-based thresholds.
Despite these successes, challenges remain. Vocabulary mismatch—particularly with unseen medical acronyms in private practice—remains a hurdle. Future work will focus on more granular diagnostic codes and optimizing the "cost" of human intervention within the abstention framework. DeepTag provides a scalable blueprint for automating medical coding in domains where structured data is currently scarce.
Summary updated