Meta-learning Pathologies from Radiology Reports using Variance Aware Prototypical Networks

Arijit Sehanobish, Kawshik Kannan, Nabila Abraham, Anasuya Das, Benjamin Odry

Introduction

The integration of Artificial Intelligence into clinical workflows is often hindered by the "data bottleneck." While Large Language Models (LLMs) have demonstrated remarkable capabilities in general natural language processing, their application to specialized medical domains—such as radiology—requires extensive fine-tuning on high-quality, labeled datasets. In radiology, generating these labels is both time-consuming and expensive, as it necessitates the expertise of board-certified physicians.

Few-shot learning, specifically meta-learning, offers a potential solution by enabling models to adapt to new tasks with minimal supervision. However, traditional few-shot methods like Prototypical Networks (ProtoNets) often struggle with the inherent variability of clinical language. Radiologists exhibit diverse dictation styles, and the linguistic "spread" of a single pathology can be significant. This paper introduces Variance Aware Prototypical Networks, a method that models medical pathologies as distributions rather than static points, significantly improving classification performance in low-data regimes.

Methodology

The proposed architecture extends the standard Prototypical Network framework by incorporating variance awareness and robust distance metrics.

1. Distributional Class Representation

Unlike vanilla ProtoNets, which represent a class by the mean vector (prototype) of its support set, Variance Aware Prototypical Networks model each class as a Gaussian distribution. This approach captures both the central tendency of a pathology's description and its inherent variance, allowing the model to account for stylistic diversity in reports.

2. The 2-Wasserstein Distance

To measure the similarity between a query embedding and a class distribution, the authors employ the 2-Wasserstein distance (also known as the Wasserstein-Bures metric). Unlike the Kullback-Leibler (KL) divergence, the Wasserstein distance is a true metric that satisfies the triangle inequality. This property provides greater stability during the optimization process and a more mathematically sound basis for comparing embeddings to distributions.

3. Regularization and Efficiency

To prevent the variance from expanding uncontrollably and to ensure tight clustering, a regularization term based on the Frobenius norm of the covariance matrices is introduced. Furthermore, the model utilizes a PubMedBERT backbone with Adapters. By freezing the majority of the transformer parameters and only training approximately 10 million adapter parameters, the system achieves high performance without the risk of catastrophic forgetting or overfitting on small medical datasets.

Experimental Results

The model was meta-trained on a large-scale internal dataset of shoulder MRI reports from 74 institutions. Evaluation was conducted across diverse internal downstream tasks and standard public NLP benchmarks.

Performance in Radiology

The inclusion of variance awareness led to substantial gains in classification accuracy. For instance, on the Cervical Foraminal Stenosis task, the Variance Aware ProtoNet achieved an F1 score of 0.91, a significant improvement over the 0.78 F1 score produced by the vanilla ProtoNet.

Generalization and Benchmarking

On public NLP benchmarks, the method outperformed "Leopard," a competitive meta-learning baseline, by 5 points on 4-shot tasks and 2 points on 16-shot tasks. These results suggest that the benefits of modeling class variance extend beyond the medical domain.

Clinical Utility and OOD Detection

Beyond classification, the distributional approach enables natural Out-of-Distribution (OOD) detection. If a report falls outside the learned variances of all known classes, the system can flag it for manual review. Additionally, the model is highly efficient, with an average latency of 68ms per report on an NVIDIA V100 GPU, making it suitable for real-time clinical integration.

Discussion and Limitations

While the model shows strong performance across most tasks, challenges remain in distinguishing between pathologies with high linguistic overlap, such as disc herniation and cord compression in the lumbar spine. In these cases, the variance in descriptions is too high for the model to achieve clear separability with only a few examples.

Future research will focus on expanding the meta-training set diversity and exploring more advanced parameter-efficient architectures. While the potential for "pseudo-labeling" to accelerate data annotation is promising, the authors emphasize the need for expert oversight to prevent the propagation of algorithmic biases in clinical settings.

Summary updated

BibTeX
@inproceedings{sehanobish2022meta,
  author = {Arijit Sehanobish and Kawshik Kannan and Nabila Abraham and Anasuya Das and Benjamin Odry},
  title = {{Meta-learning Pathologies from Radiology Reports using Variance Aware Prototypical Networks}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2022},
  doi = {10.18653/v1/2022.emnlp-industry.34},
}
Figure