Context-Aware Automatic Text Simplification of Health Materials in Low-Resource Domains

Tarek Sakakini, Jong Yoon Lee, Aditya Duri, Renato Ferreira Leitão Azevedo, Victor Sadauskas, Kuangxiao Gu, Suma Bhat, Daniel G. Morrow, James Graumlich, Saqib Walayat, Mark Hasegawa-Johnson, Thomas S. Huang, Ann Willemsen-Dunlap, Donald HalpinLOUHI @ EMNLP 2020 · DOI

Introduction

Effective communication between healthcare providers and patients is critical for safety and treatment adherence. However, a significant "health literacy gap" exists, where complex medical terminology—such as "PO" (by mouth) or "SBP" (systolic blood pressure)—often leads to patient misunderstanding and medication errors. While Automatic Text Simplification (ATS) offers a potential solution, the medical domain suffers from severe data scarcity. State-of-the-art neural models typically require large parallel corpora of professional and simplified text, which are difficult to obtain due to privacy regulations like HIPAA and proprietary concerns.

The Dr. Babel Fish (DBF) Framework

To address these challenges, researchers developed Dr. Babel Fish (DBF), an unsupervised, context-aware system designed to simplify medical instructions without requiring parallel training data. DBF utilizes a three-stage pipeline to identify and replace complex terms while maintaining semantic accuracy.

The three-stage pipeline of Dr. Babel Fish: Identification, Candidate Generation, and Decoding.

The pipeline consists of three distinct phases:

  1. Identification: The system flags "hard" words by comparing them against a frequency threshold in a patient-facing health corpus. Words that appear rarely in patient literature are targeted for simplification.
  2. Candidate Generation: The system queries the Unified Medical Language System (UMLS) to retrieve all possible synonyms and related concepts for the identified hard words.
  3. Decoding (The Context Engine): This core innovation uses a 6-gram language model trained on 11 million words of patient-friendly text. It evaluates synonym combinations to select the one that best fits the context. For example, it can correctly identify that "mg" refers to "milligrams" rather than "magnesium" based on the surrounding sentence structure.

Experimental Setup and Datasets

The study utilized two primary datasets to evaluate the system's efficacy:

  • Parallel Medication Instructions: A new dataset of 4,554 unique, de-identified medication instructions for diabetic patients, annotated with physician-verified "gold standard" simplifications.
  • Monolingual Patient Corpus: A collection of ~11 million words scraped from reputable sources like MedlinePlus and Mayo Clinic to train the language model on accessible medical prose.

Key Results and Human Impact

DBF demonstrated superior performance over existing baselines in both automated and human evaluations. In automated tests, it achieved a SARI score of 55.33, significantly outperforming the previous MetaMap+CHV standard (45.64). Unlike neural "Seq2Seq" models, which often struggle in low-resource settings and produce nonsensical repetitions or "hallucinations," DBF's knowledge-aware approach provides a reliable safety net.

Most importantly, a human study involving 160 participants revealed that DBF-simplified instructions improved comprehension accuracy by 24.4% compared to original physician-written text. This improvement was consistent across instructions with varying numbers of complex terms.

Comprehension accuracy comparison between original and simplified instructions across varying numbers of hard words.

Conclusion and Future Directions

Dr. Babel Fish represents a significant advancement in lexical simplification for the medical domain. By leveraging structured knowledge bases and context-aware decoding, it bypasses the need for massive parallel datasets that typically bottleneck medical AI. While the current system focuses on word-level replacement, future work will explore phrase-level simplification and hybrid models to address complex sentence structures (syntax), further closing the health literacy gap and improving patient safety.

Summary updated

BibTeX
@inproceedings{sakakini2020context,
  author = {Tarek Sakakini and Jong Yoon Lee and Aditya Duri and Renato Ferreira Leitão Azevedo and Victor Sadauskas and Kuangxiao Gu and Suma Bhat and Daniel G. Morrow and James Graumlich and Saqib Walayat and Mark Hasegawa-Johnson and Thomas S. Huang and Ann Willemsen-Dunlap and Donald Halpin},
  title = {{Context-Aware Automatic Text Simplification of Health Materials in Low-Resource Domains}},
  booktitle = {LOUHI @ EMNLP},
  year = {2020},
  doi = {10.18653/v1/2020.louhi-1.13},
}
Figure