Magical: Medical Lay Language Generation via Semantic Invariance and Layperson-tailored Adaptation

Weibin Liao, Tianlong Wang, Yinghao Zhu, Yasha Wang, Junyi Gao, Liantao Ma

Abstract

Medical Lay Language Generation (MLLG) is essential for making complex scientific findings accessible to the general public. While parameter-efficient fine-tuning methods like Low-Rank Adaptation (LoRA) are commonly used to adapt Large Language Models (LLMs) for this task, they often struggle with multi-source heterogeneous datasets. Standard LoRA frequently fails to maintain semantic fidelity and struggles to produce diverse layperson-friendly styles. To address these issues, we introduce Magical, an asymmetric LoRA architecture specifically designed for MLLG in heterogeneous data environments. Magical utilizes a shared matrix $A$ to capture core abstractive summarization capabilities and multiple isolated matrices $B$ to handle diverse stylistic requirements. By incorporating a Semantic Invariance Constraint and a Recommendation-guided Switch, Magical ensures semantic accuracy while allowing for tailored adaptation. Experimental results across three real-world datasets show that Magical consistently outperforms vanilla LoRA and its variants, while reducing trainable parameters by 31.66%.

Introduction: The "Lost in Translation" Problem in Medical AI

Bridging the gap between expert-level medical discourse and patient understanding is a critical challenge in healthcare communication. Medical Lay Language Generation (MLLG) aims to automate the translation of dense biomedical text into accessible language. However, the current standard for fine-tuning LLMs—Low-Rank Adaptation (LoRA)—faces significant hurdles when dealing with the diverse styles and rigorous accuracy requirements of medical data.

Researchers have identified two primary failures in standard LoRA applications for medical simplification:

  1. Semantic Drift: During the simplification process, models often "hallucinate" or lose core medical meanings. This occurs because the low-rank projection shifts the text's representation away from its original semantic space.
  2. Style Interference: Medical datasets are heterogeneous; for instance, some favor extreme brevity while others provide extensive background context. Training a single LoRA on these conflicting sources leads to "mushy," inconsistent outputs.
Visualization of semantic subspace shift between expert and lay text, illustrating the challenge of semantic drift.

The Magical Framework: Decoupling Meaning from Style

To solve these issues, the authors propose Magical, an asymmetric architecture that decouples the adaptation process into two distinct roles: content preservation and stylistic adaptation.

Shared Matrix A (The "What")

Matrix $A$ is shared across all datasets and is responsible for the universal task of abstractive summarization. To prevent semantic drift, Magical employs Semantic-Relevant Layer Identification (SRLI) to target layers critical for logical reasoning. It then applies Semantic Contrastive Learning (SCL) to force the model to keep the internal representations of "expert" and "lay" versions of a sentence close together, ensuring the core meaning remains invariant.

Isolated Matrices B (The "How")

Instead of a single $B$ matrix, Magical utilizes multiple isolated $B$ matrices, each tailored to a specific dataset or target style. This prevents style interference, allowing the model to learn that a "Cochrane" style might require brevity while an "eLife" style requires more explanatory context.

Recommendation-guided Switch

To navigate these specialized matrices, Magical incorporates an external interface called the Recommendation-guided Switch. This component prompts the LLM to activate the specific $B$ matrix that matches the target audience or dataset, rather than forcing the model to guess the appropriate style.

The Magical architecture featuring shared matrix A for semantic invariance and isolated matrices B for layperson-tailored adaptation.

Experimental Evaluation

Magical was evaluated on three distinct real-world MLLG datasets:

  • Cochrane: Summaries that typically decrease word count.
  • eLife: Summaries that often increase word count to provide background explanation.
  • PLOS Genetics: A mid-point in terms of complexity and length.

The framework's versatility was further tested on the unseen EASI dataset and even extended into the Legal domain to demonstrate its generalizability beyond medicine.

Key Results and Insights

  • Superior Performance: Magical consistently outperformed vanilla LoRA and state-of-the-art variants (such as DoRA and rsLoRA) across ROUGE and BLEU metrics.
  • Parameter Efficiency: The model achieved these results while reducing the number of trainable parameters by 31.66% compared to standard multi-LoRA setups.
  • Validation: In blind tests, both human medical experts and advanced LLMs (like DeepSeek-R1) rated Magical’s summaries as more accurate and readable than those produced by standard methods.
  • Robustness: The system proved resilient; even when the Recommendation Agent was only 60% accurate in selecting the style matrix, Magical still outperformed standard LoRA.

Conclusion

Magical represents a significant step forward in making medical information accessible without sacrificing scientific accuracy. By addressing the hidden flaws of LoRA—specifically semantic drift and style interference—the framework provides a robust solution for heterogeneous data environments. Future work will focus on developing "cold-start" recommendation agents capable of sensing a user's health literacy level in real-time to adapt medical narratives dynamically.

Summary updated

Abstract

Medical Lay Language Generation (MLLG) plays a vital role in improving the accessibility of complex scientific content for broader audiences. Recent literature to MLLG commonly employ parameter-efficient fine-tuning methods such as Low-Rank Adaptation (LoRA) to fine-tuning large language models (LLMs) using paired expert-lay language datasets. However, LoRA struggles with the challenges posed by multi-source heterogeneous MLLG datasets. Specifically, through a series of exploratory experiments, we reveal that standard LoRA fail to meet the requirement for semantic fidelity and diverse lay-style generation in MLLG task. To address these limitations, we propose Magical, an asymmetric LoRA architecture tailored for MLLG under heterogeneous data scenarios. Magical employs a shared matrix $A$ for abstractive summarization, along with multiple isolated matrices $B$ for diverse lay-style generation. To preserve semantic fidelity during the lay language generation process, Magical introduces a Semantic Invariance Constraint to mitigate semantic subspace shifts on matrix $A$. Furthermore, to better adapt to diverse lay-style generation, Magical incorporates the Recommendation-guided Switch, an externally interface to prompt the LLM to switch between different matrices $B$. Experimental results on three real-world lay language generation datasets demonstrate that Magical consistently outperforms prompt-based methods, vanilla LoRA, and its recent variants, while also reducing trainable parameters by 31.66%. Our code is publicly available at https://github.com/tianlwang/Magical.git.
BibTeX
@inproceedings{liao2025magical,
  author = {Weibin Liao and Tianlong Wang and Yinghao Zhu and Yasha Wang and Junyi Gao and Liantao Ma},
  title = {{Magical: Medical Lay Language Generation via Semantic Invariance and Layperson-tailored Adaptation}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2025},
  url = {https://openreview.net/forum?id=mH9FJi3KTX},
}
Figure