Abstract
Expert-layman text style transfer technologies have the potential to improve communication between members of scientific communities and the general public. High-quality information produced by experts is often filled with difficult jargon laypeople struggle to understand. This is a particularly notable issue in the medical domain, where layman are often confused by medical text online. At present, two bottlenecks interfere with the goal of building high-quality medical expert-layman style transfer systems: a dearth of pretrained medical-domain language models spanning both expert and layman terminologies and a lack of parallel corpora for training the transfer task itself. To mitigate the first issue, we propose a novel language model (LM) pretraining task, Knowledge Base Assimilation, to synthesize pretraining data from the edges of a graph of expert- and layman-style medical terminology terms into an LM during self-supervised learning. To mitigate the second issue, we build a large-scale parallel corpus in the medical expert-layman domain using a margin-based criterion. Our experiments show that transformer-based models pretrained on knowledge base assimilation and other well-established pretraining tasks fine-tuning on our new parallel corpus leads to considerable improvement against expert-layman transfer benchmarks, gaining an average relative improvement of our human evaluation, the Overall Success Rate (OSR), by 106%. We release our code and parallel corpus for future research.
Introduction
In the medical domain, a significant "knowledge gap" exists between healthcare professionals and the general public. While experts communicate using precise, technical jargon (e.g., "myocardial infarction"), laypeople often use descriptive, common language (e.g., "heart attack"). This discrepancy hinders effective information retrieval and patient understanding.
Technically, bridging this gap is a Text Style Transfer (TST) problem. However, unlike sentiment-based TST, expert-layman transfer requires deep domain-specific knowledge. Current systems face two primary bottlenecks: a lack of parallel datasets where identical concepts are expressed in both styles, and the inability of standard language models to align technical terms with their layman counterparts. This paper introduces a novel self-supervised pretraining task and a large-scale mined parallel corpus to address these challenges.
Methodology
Knowledge Base Assimilation (KBA)
The core innovation of this work is Knowledge Base Assimilation (KBA), a self-supervised pretraining task designed to inject terminological knowledge directly into the language model. Using the Unified Medical Language System (UMLS), the researchers constructed a "Terminology Bijective Graph" that maps expert terms to layman phrases.
As illustrated in Figure 2, the KBA process involves identifying a medical term in a sentence and replacing it with its counterpart from the opposite style. The model is then tasked with reconstructing the original sentence. This forces the transformer to learn semantic mappings between styles within a contextual framework. KBA is integrated into a multi-task pretraining ensemble that includes standard Masked Language Modeling (Mask), sentence un-shuffling (Switch), and gap-filling (Delete).
Parallel Corpus Mining
To overcome the scarcity of parallel data, the authors utilized a margin-based criterion with LASER embeddings to mine high-quality sentence pairs from the Merck Manuals (MSD) dataset. This resulted in a new parallel corpus of 11,512 pairs, providing the necessary supervision for fine-tuning the style transfer task.
Experimental Results
Evaluation Metrics
The models were evaluated using automated metrics (BLEU, Style Accuracy, Perplexity) and extensive human evaluation via Amazon Mechanical Turk. Human raters assessed outputs based on Content Success Rate (CSR), Understanding Success Rate (USR), Grammatical Success Rate (GSR), and an Overall Success Rate (OSR).
Performance Analysis
The proposed KBA + SSL Large model demonstrated superior performance across nearly all human evaluation metrics. As shown in Figure 4, it achieved an OSR of 0.216, representing a 75.6% improvement over the strongest baseline, DeleteAndRetrieve (OSR 0.123). Across all human evaluations, the model showed an average relative improvement of 106%.
Qualitative analysis revealed that the KBA-enhanced models successfully translated complex phrases like "crackles on auscultation" into accurate layman terms like "airway narrowing," whereas baseline models often produced irrelevant hallucinations. Furthermore, the model exhibited high data efficiency; even when trained on only 40% of the mined parallel data, it outperformed baselines trained on the full dataset.
Conclusion
By combining self-supervised Knowledge Base Assimilation with a large-scale mined parallel corpus, this research significantly advances the state-of-the-art in medical expert-layman style transfer. While the current work focuses on the medical domain, the KBA framework is potentially generalizable to other technical fields. Future work will explore more complex knowledge graph structures and address the critical safety requirements for deploying such systems in clinical settings.
Summary updated