Attentive Multiview Text Representation for Differential Diagnosis

Hadi Amiri, Mitra Mohtarami, Isaac S. KohaneACL 2021 · OpenReview

TL;DR

This paper introduces an "Attentive Multiview Neural Model" (AMNM) to improve differential diagnosis by combining different types of patient and disease information. By intelligently weighing and fusing these diverse data "views," the model significantly outperforms existing methods in ranking the most likely diseases for patients with rare and undiagnosed conditions.

What Problem Does This Paper Address?

Diagnosing rare and undiagnosed diseases is a significant challenge in healthcare. Electronic Health Records (EHRs) contain a wealth of information, but their complexity, missing data, and imprecise descriptions make it difficult to extract actionable insights. Specifically, distinguishing between diseases with similar clinical features is a complex task that often requires expert knowledge. This paper tackles the problem of differential diagnosis by framing it as a ranking task: given a patient's medical history, the goal is to rank potential diseases from most to least probable. The challenge lies in effectively leveraging all available information, which can come in various forms (e.g., free text, coded medical concepts), and determining which pieces of information are most relevant for accurate diagnosis.

Methods

The core innovation is the Attentive Multiview Neural Model (AMNM). This model works by:

  1. Multiview Representation: It takes patient and disease descriptions from multiple "views." For instance, one view might be the raw text of a medical history, while another could be structured medical concepts extracted from that text.
  2. Attention Mechanism: An attention sub-network learns to assign "attentive weights" to each view. This means the model dynamically decides how much importance to give to each type of information (e.g., text vs. codes) for a given patient-disease pair. This is achieved by feeding query (patient) and document (disease) embeddings into a feed-forward network that outputs these weights.
  3. Fusion Network: After the attention weights are determined, a fusion sub-network combines the weighted representations from each view. The paper explores different fusion strategies, including dot product, outer product, and 1D convolution, to capture complex interactions between query and document features within each view.
  4. Ranking: Finally, a softmax layer uses the fused representations to estimate the relevance score between the patient and each potential disease, allowing for a ranked list of diagnoses.

The AMNM integrates traditional and recent text representation techniques, notably using BERT for text and SVMs for coded concepts.

Datasets

The study utilizes data from the Undiagnosed Diseases Network (UDN). This dataset includes:

  • Medical histories of 257 patients with severe clinical conditions.
  • General descriptions of over 9,000 diseases from the Online Mendelian Inheritance in Man (OMIM) dataset.
  • Relevance judgments between patients and diseases provided by expert clinicians from the UDN.

The dataset was split into training (80%), validation (10%), and testing (10%) sets. For each positive patient-disease pair in the training set, a negative pair was created. At test time, all possible patient-disease combinations were generated (over 218K pairs) for ranking.

Key Insights and Ideas

  • The Power of Diverse Data Views: The paper strongly suggests that combining different types of information (textual descriptions and structured medical codes) is crucial for improving diagnostic accuracy.
  • Attention is Key: Simply concatenating information from different views isn't enough. The attention mechanism, by learning to prioritize more informative views, significantly boosts performance. This is particularly important when one view might be more reliable or informative than another for a specific case.
  • Fusion Matters: The way information from different views is combined (fused) also impacts results. The paper found that simpler fusion methods like dot and outer products were more effective than a more complex convolutional approach, possibly due to the latter's reliance on average pooling which might dilute important features.
  • BERT's Nuances in Clinical Text: While BERT excels, its performance can be affected by how medical concepts are represented. Replacing specific medical terms with their "preferred" UMLS concept (as done in one of the experiments) can sometimes make the text less coherent, potentially hindering BERT's effectiveness.

Key Results

  • The AMNM model, particularly when using BERT for text and SVMs for code views with a dot product fusion (AMNMbert-sums (gdot)), significantly outperformed the best-performing baseline (BERT on text view) by 3.4, 5.8, and 6.2 points in MAP, P@5, and P@10 metrics, respectively.
  • The model achieved higher P@5 and P@10 scores than the commercial PhenoTips tool, which is important for clinical practice where top-ranked results are critical for diagnosis.
  • The attention sub-network was found to be 57.7% accurate in prioritizing the better view for a given patient-disease pair.
  • In cases where SVMs (code view) outperformed BERT (text view), the attention network correctly assigned higher weight to SVMs in only 0.9% of these examples, indicating room for improvement in attention accuracy.

Limitations and Future Work

  • Attention Accuracy: The authors acknowledge that the attention mechanism's accuracy in prioritizing the better view could be improved, especially in cases where the code view is superior. This is identified as a key area for future work.
  • Feature Enrichment: The model's performance could be further enhanced by incorporating more specific patient and disease information, such as demographic data and clinical synopses.
  • Word Distribution Differences: Addressing the differences in word distributions between patient queries and disease documents is another avenue for future research.
  • Practical Evaluation: The paper highlights that for rare diseases, metrics like Hit@K (measuring if any relevant disease is in the top K results) are more clinically relevant than standard metrics. Their model achieved a Hit@20 of 0.49, compared to 0.37 for the best baseline.

Summary updated

Abstract

We present a text representation approach that can combine different views (representations) of the same input through effective data fusion and attention strategies for ranking purposes. We apply our model to the problem of differential diagnosis, which aims to find the most probable diseases that match with clinical descriptions of patients, using data from the Undiagnosed Diseases Network. Our model outperforms several ranking approaches (including a commercially-supported system) by effectively prioritizing and combining representations obtained from traditional and recent text representation techniques. We elaborate on several aspects of our model and shed light on its improved performance.
BibTeX
@inproceedings{amiri2021attentive,
  author = {Hadi Amiri and Mitra Mohtarami and Isaac S. Kohane},
  title = {{Attentive Multiview Text Representation for Differential Diagnosis}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2021},
  url = {https://openreview.net/forum?id=y1a1Pg8or7},
}
Figure