Abstract
Medical Lay Language Generation (MLLG) is essential for making complex scientific findings accessible to the general public. While parameter-efficient fine-tuning methods like Low-Rank Adaptation (LoRA) are commonly used to adapt Large Language Models (LLMs) for this task, they often struggle with multi-source heterogeneous datasets. Standard LoRA frequently fails to maintain semantic fidelity and struggles to produce diverse layperson-friendly styles. To address these issues, we introduce Magical, an asymmetric LoRA architecture specifically designed for MLLG in heterogeneous data environments. Magical utilizes a shared matrix $A$ to capture core abstractive summarization capabilities and multiple isolated matrices $B$ to handle diverse stylistic requirements. By incorporating a Semantic Invariance Constraint and a Recommendation-guided Switch, Magical ensures semantic accuracy while allowing for tailored adaptation. Experimental results across three real-world datasets show that Magical consistently outperforms vanilla LoRA and its variants, while reducing trainable parameters by 31.66%.
Introduction: The "Lost in Translation" Problem in Medical AI
Bridging the gap between expert-level medical discourse and patient understanding is a critical challenge in healthcare communication. Medical Lay Language Generation (MLLG) aims to automate the translation of dense biomedical text into accessible language. However, the current standard for fine-tuning LLMs—Low-Rank Adaptation (LoRA)—faces significant hurdles when dealing with the diverse styles and rigorous accuracy requirements of medical data.
Researchers have identified two primary failures in standard LoRA applications for medical simplification:
- Semantic Drift: During the simplification process, models often "hallucinate" or lose core medical meanings. This occurs because the low-rank projection shifts the text's representation away from its original semantic space.
- Style Interference: Medical datasets are heterogeneous; for instance, some favor extreme brevity while others provide extensive background context. Training a single LoRA on these conflicting sources leads to "mushy," inconsistent outputs.
The Magical Framework: Decoupling Meaning from Style
To solve these issues, the authors propose Magical, an asymmetric architecture that decouples the adaptation process into two distinct roles: content preservation and stylistic adaptation.
Shared Matrix A (The "What")
Matrix $A$ is shared across all datasets and is responsible for the universal task of abstractive summarization. To prevent semantic drift, Magical employs Semantic-Relevant Layer Identification (SRLI) to target layers critical for logical reasoning. It then applies Semantic Contrastive Learning (SCL) to force the model to keep the internal representations of "expert" and "lay" versions of a sentence close together, ensuring the core meaning remains invariant.
Isolated Matrices B (The "How")
Instead of a single $B$ matrix, Magical utilizes multiple isolated $B$ matrices, each tailored to a specific dataset or target style. This prevents style interference, allowing the model to learn that a "Cochrane" style might require brevity while an "eLife" style requires more explanatory context.
Recommendation-guided Switch
To navigate these specialized matrices, Magical incorporates an external interface called the Recommendation-guided Switch. This component prompts the LLM to activate the specific $B$ matrix that matches the target audience or dataset, rather than forcing the model to guess the appropriate style.
Experimental Evaluation
Magical was evaluated on three distinct real-world MLLG datasets:
- Cochrane: Summaries that typically decrease word count.
- eLife: Summaries that often increase word count to provide background explanation.
- PLOS Genetics: A mid-point in terms of complexity and length.
The framework's versatility was further tested on the unseen EASI dataset and even extended into the Legal domain to demonstrate its generalizability beyond medicine.
Key Results and Insights
- Superior Performance: Magical consistently outperformed vanilla LoRA and state-of-the-art variants (such as DoRA and rsLoRA) across ROUGE and BLEU metrics.
- Parameter Efficiency: The model achieved these results while reducing the number of trainable parameters by 31.66% compared to standard multi-LoRA setups.
- Validation: In blind tests, both human medical experts and advanced LLMs (like DeepSeek-R1) rated Magical’s summaries as more accurate and readable than those produced by standard methods.
- Robustness: The system proved resilient; even when the Recommendation Agent was only 60% accurate in selecting the style matrix, Magical still outperformed standard LoRA.
Conclusion
Magical represents a significant step forward in making medical information accessible without sacrificing scientific accuracy. By addressing the hidden flaws of LoRA—specifically semantic drift and style interference—the framework provides a robust solution for heterogeneous data environments. Future work will focus on developing "cold-start" recommendation agents capable of sensing a user's health literacy level in real-time to adapt medical narratives dynamically.
Summary updated