TL;DR
This paper tackles the challenge of making Japanese medical terminology understandable to patients by constructing a word complexity lexicon. They achieved this by crowdsourcing annotations from non-medical professionals and training a machine learning model that significantly outperforms previous methods, paving the way for better patient-doctor communication.
What Problem Does This Paper Address?
Effective communication between medical professionals and patients is crucial for understanding diagnoses and treatment plans. A major barrier to this communication is the inherent complexity of medical terminology, which is often difficult for laypeople to grasp. While medical text simplification has been an active research area in English, Japanese has lagged due to a lack of readily available language resources. This paper aims to bridge this gap by creating a lexicon that quantifies the complexity of Japanese medical terms from a patient's perspective.
Methods
The core of this work involves two main stages: annotation and model building.
First, they conducted a crowdsourcing effort to gather complexity annotations for 10,000 Japanese medical terms. To ensure diversity, they recruited 40 annotators segmented by age and gender. These annotators were asked to rate terms on a five-point scale, from "used in daily conversation" to "never seen or heard of." Rigorous quality control measures, including initial small annotations and inter-annotator agreement checks (Quadratic Weighted Kappa), were implemented to ensure data reliability.
Second, they trained a machine learning model to predict word complexity. They experimented with various features, including:
- Basic Features: Character types (hiragana, katakana, kanji, numbers, alphabetic), number of morphemes, and frequencies of characters and morphemes within a Japanese Wikipedia corpus.
- Proposed Features:
- PF1: Frequencies on Web Corpus: Character and morpheme frequencies from the large-scale CC-100 web corpus.
- PF2: Word Frequencies: Frequencies of medical terms in word units, using a Japanese disease lexicon to extend MeCab's analysis.
- PF3: Word Embeddings: Pre-trained fastText embeddings.
A Support Vector Machine (SVM) with an RBF kernel was chosen as the classification model, with hyperparameters tuned via grid search and cross-validation.
Datasets
- Annotated Medical Terms: 10,000 Japanese medical terms randomly selected from the top 30,000 terms in a large-scale Japanese medical terminology lexicon (J-MeDic).
- Japanese Wikipedia Corpus: Used for calculating basic character and morpheme frequencies.
- CC-100 Corpus: A large-scale web corpus used for calculating character and morpheme frequencies (Proposed Feature PF1).
- Japanese Disease Lexicon: Used to extend MeCab for word unit frequency calculation (Proposed Feature PF2).
The paper also mentions the creation of a word complexity lexicon covering approximately 370,000 Japanese medical terms, which will be made available upon publication.
Key Insights and Ideas
- Age Matters, But Complexity Persists: While understanding of medical terms increases with age, even individuals in their 50s struggle with over 70% of medical terms, highlighting a persistent communication gap.
- Gendered Knowledge Gaps: Young men in their 30s and younger showed unfamiliarity with terms related to pregnancy and childbirth, suggesting gender-specific knowledge differences.
- Word Embeddings are Crucial: The ablation study revealed that excluding word embeddings (PF3) led to a significant drop in performance, underscoring their importance for capturing semantic nuances related to complexity.
- Web Corpus Frequencies are Informative: Even using frequency features from a large-scale web corpus (PF1) alone outperformed baseline methods, demonstrating the value of broad linguistic data.
Key Results
The proposed word complexity estimation model achieved a significant improvement over existing methods:
- Accuracy: 0.793 (compared to 0.653 for baselines).
- Quadratic Weighted Kappa (QWK): 0.732 (compared to 0.456 for baselines).
The model demonstrated the utility of all proposed features, with word embeddings (PF3) being particularly impactful. The final model, utilizing a combination of basic and proposed features, achieved approximately 80% accuracy in classifying five levels of medical term complexity.
Limitations and Future Work
- Patient-Centricity: The authors acknowledge that the "word complexity" was judged by patients themselves. While this is the goal, it's possible that even terms deemed simple by patients could still lead to medical misunderstandings.
- Scope of Terminology: The current study focused on disease and symptom names. Future work aims to extend complexity estimation to other medical domains, such as drug names and anatomical terms.
- Potential for Misinterpretation: The paper implicitly suggests that even if a term is understood, its precise medical meaning might still be misconstrued, a nuance not fully captured by a simple complexity score.
Summary updated