Matching-Based Term Semantics Pre-Training for Spoken Patient Query Understanding

Zefa Hu, Xiuyi Chen, Haoran Wu, Minglun Han, Ziyi Ni, Jing Shi, Shuang Xu, Bo Xu

Introduction

In medical dialogue systems, the ability to accurately interpret patient queries is a fundamental challenge. This process, known as Medical Slot Filling (MSF), involves extracting structured information—such as symptoms, medications, or body parts—from natural language. However, a significant semantic gap exists between the colloquial language used by patients (e.g., "my stomach feels like it's on fire") and the formal terminology used in clinical databases (e.g., Gastritis or Heartburn).

Traditional approaches to MSF typically treat the problem as a multi-label classification task or a generative task. Classification models often struggle with the vast and evolving nature of medical labels, especially in low-resource settings. Generative models, while flexible, are prone to hallucinations and error propagation. To address these limitations, researchers have introduced the Term Semantics Pre-trained Matching Network (TSPMN), which reframes MSF as a matching problem and utilizes specialized pre-training to bridge the gap between patient slang and medical terms.

Methodology

The Matching Paradigm

Instead of asking a model to classify a query into one of hundreds of fixed categories, TSPMN treats the task as a matching exercise. The model is provided with both the patient's query and a set of candidate medical terms. By modeling the interaction between the query and the terms directly, the model treats the terminology as external knowledge, reducing the need for the network to memorize every possible mapping within its weights.

Architecture: TSPMN

The TSPMN architecture utilizes a Transformer-based encoder, initialized with BERT, to process the joint input of the query and the candidate terms. This cross-attention mechanism allows the model to capture nuanced relationships between the informal description and the formal medical label.

Specialized Pre-training Tasks

To enhance the model's understanding of medical semantics before fine-tuning on specific datasets, the authors introduced two self-supervised pre-training tasks using over 3.5 million unlabeled medical dialogues:

  1. Contrastive Term Discrimination (CTD): The model is presented with a dialogue and a mix of relevant and random medical terms. It must learn to distinguish which terms are semantically linked to the conversation, improving its ability to recognize medical concepts in context.
  2. Matching-based Mask Term Modeling (MMTM): This task involves masking medical terms within a dialogue. The model must then predict the missing term by matching the surrounding context against a comprehensive terminology database. This aligns the pre-training objective closely with the downstream matching task.

Experimental Results

The TSPMN model was evaluated against two major Chinese medical benchmarks: the Medical Slot Filling (MSL) dataset and the Medical Dialogue (MedDG) dataset.

Performance Benchmarks

TSPMN achieved new state-of-the-art (SOTA) results across both datasets. On the MSL benchmark, the model reached an accuracy of 83.4%, significantly outperforming established baselines such as PromptGen (79.6%) and standard BERT-Raw (70.9%).

Low-Resource and Few-Shot Success

The most striking advantage of the matching-based approach is its robustness in data-scarce scenarios. In a 1-shot learning environment—where the model sees only one example per category—TSPMN achieved 55.78% accuracy. In contrast, the standard BERT-Raw baseline managed only 4.56%. This suggests that the matching paradigm allows the model to generalize its understanding of term relationships rather than relying on rote memorization of labels.

Ablation Study

Ablation experiments confirmed that the specialized pre-training tasks (CTD and MMTM) were critical to the model's success. Removing these components led to a significant decline in performance, demonstrating that general-purpose language models like BERT require domain-specific "matching" alignment to handle medical dialogues effectively.

Conclusion and Future Work

TSPMN represents a significant step forward in making medical AI more accessible and accurate for patients. By shifting from classification to matching and employing targeted pre-training, the model effectively bridges the gap between colloquialisms and formal medicine.

One current limitation noted by the authors is the model's inability to account for the "polarity" of a term—for instance, distinguishing between a patient saying they have a "persistent cough" versus "no cough." Future research will focus on integrating negation and severity detection into the matching framework to provide a more comprehensive understanding of patient health status.

Summary updated

BibTeX
@inproceedings{hu2023matching,
  author = {Zefa Hu and Xiuyi Chen and Haoran Wu and Minglun Han and Ziyi Ni and Jing Shi and Shuang Xu and Bo Xu},
  title = {{Matching-Based Term Semantics Pre-Training for Spoken Patient Query Understanding}},
  booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing},
  year = {2023},
  doi = {10.1109/icassp49357.2023.10096604},
}
Figure