TL;DR
This paper explores automatic text simplification for Spanish medical documents, a crucial step for patient comprehension. The authors found that combining a specialized medical lexicon with deep learning models, particularly mBART, significantly improved simplification quality, especially in semantic accuracy. Their work introduces a valuable Spanish medical lexicon and models, contributing to more accessible health information.
What Problem Does This Paper Address?
Medical texts are notoriously dense with technical jargon and complex sentence structures, creating a significant barrier for patients trying to understand their health conditions. While automatic text simplification (ATS) aims to bridge this gap, ensuring that simplified text is both accurate and easy to understand, especially in specialized domains like medicine, remains a challenge. This paper specifically addresses the need for effective ATS for medical texts in Spanish, a language with fewer readily available resources compared to English. The authors aim to evaluate different ATS strategies, including lexicon-based, deep learning-based, and hybrid approaches, to find the most effective methods for this critical task.
Methods
The researchers investigated three main categories of ATS methods:
- Lexicon-based Simplification: They developed a Spanish medical lexicon, "SimpMedLexSp," containing over 14,000 pairs of technical terms and their layperson equivalents (synonyms, definitions, or paraphrases). This lexicon was used for direct lexical substitution or to append definitions/paraphrases.
- Deep Learning (DL)-based Simplification: They fine-tuned several state-of-the-art neural network models, including multilingual BART (mBART), mT5, and Pegasus XSUM, on medical text datasets. They also explored prompt-learning techniques using a BERTIN model.
- Hybrid Approaches: These methods combined the lexicon with DL models. This involved using the lexicon for fine-tuning the DL models, or for pre- and post-processing the model's output.
Datasets
The study utilized two primary datasets:
- CLARA-MeD Corpus: This corpus provided 3800 parallel technical and layperson sentences, semi-automatically aligned and revised by linguists.
- Manually Simplified Sentences: An additional 1200 parallel sentences were created by linguists from clinical trial announcements, featuring both syntactically simplified and syntactically + lexically simplified versions.
The SimpMedLexSp lexicon itself, containing over 14,000 term pairs, also served as a crucial resource.
Key Insights and Ideas
- Lexicon is Key for Medical Accuracy: The study strongly suggests that a specialized medical lexicon is vital for maintaining semantic correctness in simplified medical texts. While DL models can generate fluent text, they sometimes "hallucinate" or misinterpret medical terms, a risk mitigated by lexicon integration.
- Hybrid Approaches Outperform: Combining a lexicon with DL models, especially for fine-tuning, consistently yielded better results than using either approach in isolation.
- Human Evaluation Matters: Quantitative metrics like BLEU and SARI didn't always align with human judgment. For instance, mBART scored highly on automatic metrics but was rated lower by humans for fluency and grammaticality compared to lexicon-based or prompt-learning methods.
- Prompt Learning Offers Balance: The BERTIN prompt-learning model, particularly when fine-tuned with the lexicon, demonstrated a good balance across all evaluation aspects (grammar, adequacy, simplification), making it a strong contender.
- Manually Prepared Datasets Shine: The experiments indicated that datasets manually prepared by professionals (like the 1200 sentences) led to higher quantitative and qualitative simplification scores, suggesting their value for training high-quality models.
Key Results
- Quantitative: The mBART model, fine-tuned with the lexicon and post-processed, achieved the highest SARI scores on the 3800-sentence dataset. The lexicon-only approach performed well, outperforming mT5 and Pegasus XSUM. On the 1200 manually simplified sentences, quantitative metrics were generally higher across all models.
- Qualitative (Human Evaluation):
- For the 3800-sentence dataset, the lexicon approach and the BERTIN prompt-learning model received the highest scores for semantic correctness and adequacy.
- For the 1200-sentence dataset, the lexicon approach led in simplification, while the prompt-learning strategy showed balanced scores across all aspects.
- The BERTIN prompt-learning model fine-tuned with the lexicon was rated highly for grammaticality/fluency and simplification.
- The mBART models, despite good quantitative scores, sometimes produced hallucinations or incorrect paraphrases, as noted in human evaluations.
Limitations and Future Work
The authors acknowledge several limitations:
- Synonym Selection: They did not implement procedures for ranking or selecting the best synonyms, which can be context-dependent (e.g., "sintomático" can have different meanings).
- Lexicon Coverage and Updates: The lexicon's coverage needs expansion and regular updates with new concepts. Incorporating corpus-based frequencies could also enhance it.
- Syntactic Simplification: While lexical simplification was a focus, more advanced syntactic simplification techniques are needed to handle complex sentence structures effectively.
- Multilingual Replication: The findings need to be confirmed with similar experiments in other languages, given the scarcity of patient-oriented lexicons.
Future work could involve exploring more sophisticated synonym selection, expanding the lexicon, and integrating more advanced syntactic simplification methods. The authors also emphasize the need for further experiments with larger corpora and ideally, replication with data from other languages.
Summary updated