Abstract
This research introduces a specialized Machine Translation (MT) approach for the medical domain that incorporates "code-switching"—the practice of mixing English technical terms into Thai sentences. By fine-tuning the NLLB (No Language Left Behind) model on a custom-built dataset, the authors created a system that medical professionals significantly prefer over standard monolingual translations. The study presents a novel "Pseudo-translation Masking and Generation" pipeline and the first benchmark dataset for medical English-Thai code-switched translation.
Introduction
In the medical field, precision is a matter of life and death. However, standard machine translation tools often struggle with Thai medical contexts because many English technical terms lack direct Thai equivalents, or their "correct" translations are too obscure for clinical use. For example, a term like "hyperinflation" might be incorrectly translated using the Thai word for economic inflation, destroying the factual utility of the text.
Thai medical professionals naturally solve this by code-switching: maintaining technical English keywords while using Thai for the surrounding sentence structure. This paper aims to bridge the gap between how machines translate (monolingual) and how experts actually communicate (code-switched).
Methodology
To address the lack of existing code-switched training data, the authors proposed a "Pseudo-translation Masking and Generation" pipeline:
- Keyword Identification: GPT-4 identifies critical medical keywords (symptoms, drugs, anatomy) in English source sentences.
- Masking: These keywords are replaced with unique placeholders.
- Translation: The masked sentence is translated into Thai using a standard MT model.
- Re-insertion: The original English keywords are swapped back into the Thai sentence, creating a "pseudo-code-switched" (pseudo-CS) output.
This method was used to generate a large-scale dataset to fine-tune several variants of the NLLB-200 model.
Datasets
The study introduces a new benchmark dataset specifically designed for medical English-Thai code-switched translation:
- Training Set: Approximately 64,000 English-to-Thai CS translation pairs.
- Test Set: 1,100 pairs, rigorously verified and corrected by human annotators and Medical Doctors (MDs) to ensure factual accuracy and natural flow.
- Source Material: 10,000 medical excerpts focused on differential diagnosis and multiple-choice medical questions.
Results and Discussion
The researchers evaluated 52 different model configurations using both automated metrics and human evaluation from 23 Medical Doctors.
Human Preference and Factual Accuracy
Traditional MT metrics like BLEU and chrF correlate poorly with the preferences of medical professionals. While Google NMT often scores high on automated metrics, it frequently "over-translates" technical terms, leading to lower human preference scores compared to code-switched models.
The fine-tuned NLLB-1 model achieved a Glicko rating of 1549.55, significantly outperforming Google NMT (1398.61) and GPT-3.5 (1316.40). In terms of factual accuracy, the GPT-4-CS + Mask configuration achieved the highest score of 6.000 on a 7-point scale.
The Power of Masking
A key insight is that simply preventing a model from attempting to translate specific keywords (the "Mask" approach) significantly boosts factual accuracy across almost all model architectures, including large language models like GPT-4.
Conclusion
This research demonstrates that code-switching is a viable and preferred strategy for medical machine translation in Thai. While limitations exist regarding prompt sensitivity and hyperparameter tuning, the results provide a strong foundation for future work. Expanding this methodology to other language pairs where technical code-switching is common, such as English-Arabic or English-Spanish, represents a promising direction for professional domain translation.
Summary updated