Composition-driven symptom phrase recognition for Chinese medical consultation corpora

Xuan Gu, Zhengya Sun, Wensheng ZhangBMC Medical Informatics Decis. Mak 2021 · OpenReview

Abstract

Symptom phrase recognition is essential to improve the use of unstructured medical consultation corpora for the development of automated question answering systems. A majority of previous works typically require enough manually annotated training data or as complete a symptom dictionary as possible. However, when applied to real scenarios, they will face a dilemma due to the scarcity of the annotated textual resources and the diversity of the spoken language expressions. In this paper, we propose a composition-driven method to recognize the symptom phrases from Chinese medical consultation corpora without any annotations. The basic idea is to directly learn models that capture the composition, i.e., the arrangement of the symptom components (semantic units of words). We introduce an automatic annotation strategy for the standard symptom phrases which are collected from multiple data sources. In particular, we combine the position information and the interaction scores between symptom components to characterize the symptom phrases. Equipped with such models, we are allowed to robustly extract symptom phrases that are not seen before. Without any manual annotations, our method achieves strong positive results on symptom phrase recognition tasks. Experiments also show that our method enjoys great potential with access to plenty of corpora. Compositionality offers a feasible solution for extracting information from unstructured free text with scarce labels.

The Challenge of Unstructured Medical Data

The rise of online medical consultations has created a massive amount of unstructured data. Extracting symptoms from this data is the first step for automated medical Q&A systems. However, researchers face two major hurdles:

  1. The Data Bottleneck: Traditional Named Entity Recognition (NER) models (like BERT-CRF) require large amounts of manually labeled data, which is expensive and requires medical expertise.
  2. The Expression Gap: Patients use diverse, colloquial language (e.g., "my chest feels like it's being squeezed") that rarely matches the formal terms found in standard medical dictionaries (e.g., "chest distress").
The architecture of medical QA systems.

Existing dictionary-matching methods are too rigid, while supervised deep learning models struggle when labeled data is scarce. This paper seeks a middle ground: a way to "understand" the structure of symptoms to find them in the wild without human-labeled training sets.

The ComD Methodology

The authors propose the Composition Driven (ComD) framework, which treats symptom phrases as a combination of smaller semantic units. The workflow consists of three main modules:

The workflow of the ComD framework.
  • Position Modeling: The system automatically tags a large, standard symptom dictionary using the BIES scheme (Begin, Intermediate, End, Single). It then calculates the probability of specific words appearing in these positions. For example, the word "intermittent" might have a high probability of being a "Begin" (B) component.
  • Semantic Interaction: To determine if two words "belong" together, the model uses Word2Vec to create word embeddings. It then calculates an interaction score based on the Jensen-Shannon Divergence (JSD) of the words' neighborhoods in the embedding space. If two words share similar "neighbors" (contexts), they are more likely to be part of the same symptom phrase.
  • Compositionality Judgement: Finally, the model combines the position probabilities and interaction scores to evaluate candidate phrases in unstructured text. It uses a utility function to decide whether to keep or discard intermediate words, allowing it to "stitch together" disjointed components found in a sentence.

Experimental Setup and Datasets

The study utilizes two primary data sources:

  • Symptom Dictionary: A massive collection of 732,855 symptom phrases aggregated from medical publications, electronic health records, and web resources. This serves as the "knowledge base" for the unsupervised learning.
  • MedConSult: A dataset of nearly 1,000 real-world medical consultation records crawled from online platforms. Human-annotated labels were used only for evaluating the model's performance, not for training it.

Key Results and Insights

ComD was compared against several baselines, including dictionary-matching and supervised models like BERT-CRF and BiLSTM-CRF (trained via distant supervision on the dictionary).

  • Superior Recall: ComD significantly outperformed BERT-CRF in Recall and F1-score. While BERT often had higher Precision, it missed many colloquial variations that ComD was able to "reconstruct" through composition.
  • Ablation Success: Removing either the position modeling or the semantic interaction caused performance to plummet, proving that both "where a word goes" and "who a word hangs out with" are essential for recognition.
  • IoU Performance: At an Intersection over Union (IoU) threshold of 0.6, ComD achieved a Macro-F1 of 41.02% on the full MedConSult set, nearly doubling the performance of the BERT-CRF baseline (24.52%) in that specific unsupervised setting.

Conclusion and Future Work

The core insight of this research is that while a specific phrase might be new, its components and their arrangements are often predictable. By learning the "grammar" of symptoms, the model generalizes better than a simple dictionary lookup. The method proves that competitive results can be achieved in specialized domains without the "crutch" of expensive manual labeling.

However, limitations remain. ComD struggles with Out-of-Vocabulary (OOV) issues and non-decomposable idioms (like four-character Chinese medical idioms). Future work will explore more sophisticated design patterns for word composition and integrating the system into full computer-assisted diagnosis pipelines.

Summary updated

Abstract

Symptom phrase recognition is essential to improve the use of unstructured medical consultation corpora for the development of automated question answering systems. A majority of previous works typically require enough manually annotated training data or as complete a symptom dictionary as possible. However, when applied to real scenarios, they will face a dilemma due to the scarcity of the annotated textual resources and the diversity of the spoken language expressions. In this paper, we propose a composition-driven method to recognize the symptom phrases from Chinese medical consultation corpora without any annotations. The basic idea is to directly learn models that capture the composition, i.e., the arrangement of the symptom components (semantic units of words). We introduce an automatic annotation strategy for the standard symptom phrases which are collected from multiple data sources. In particular, we combine the position information and the interaction scores between symptom components to characterize the symptom phrases. Equipped with such models, we are allowed to robustly extract symptom phrases that are not seen before. Without any manual annotations, our method achieves strong positive results on symptom phrase recognition tasks. Experiments also show that our method enjoys great potential with access to plenty of corpora. Compositionality offers a feasible solution for extracting information from unstructured free text with scarce labels.
BibTeX
@inproceedings{gu2021composition,
  author = {Xuan Gu and Zhengya Sun and Wensheng Zhang},
  title = {{Composition-driven symptom phrase recognition for Chinese medical consultation corpora}},
  booktitle = {BMC Medical Informatics Decis. Mak},
  year = {2021},
  url = {https://openreview.net/forum?id=fwTRQssWeQ},
}
Figure