TL;DR
This paper introduces VietMed, the largest publicly available Vietnamese medical speech recognition dataset to date, addressing a critical data scarcity in this domain. They also release novel pre-trained and fine-tuned models that significantly outperform existing benchmarks, demonstrating the effectiveness of their approach even without in-domain pre-training.
What Problem Does This Paper Address?
Automatic Speech Recognition (ASR) systems thrive on vast amounts of data, but the medical domain, especially for less-resourced languages like Vietnamese, suffers from a significant lack of publicly available speech datasets. This scarcity hinders research and development, making it difficult to build accurate ASR systems for medical applications. Existing medical ASR datasets are often small, limited in scope (e.g., focusing on specific diseases or accents), or lack publicly released models, making reproducibility a challenge. Furthermore, general-purpose Vietnamese ASR models often struggle with the specialized vocabulary and nuances of medical conversations.
Methods
The authors tackle this problem by creating a comprehensive dataset and developing advanced ASR models.
- Dataset Creation: They collected a large corpus of Vietnamese speech, including:
- VietMed-L: 16 hours of labeled medical speech.
- VietMed-U: 1000 hours of unlabeled medical speech.
-
Viet-U: 1200 hours of unlabeled general-domain speech. The labeled data was meticulously annotated using a computer-assisted workflow to ensure high quality and efficiency. They also ensured broad coverage across ICD-10 disease groups and all Vietnamese accents.
-
Model Development: They focused on leveraging pre-trained models and fine-tuning them for the medical domain.
- Pre-training: They developed
w2v2-VietandXLSR-53-Vietmodels.w2v2-Vietwas trained from scratch on their general-domain data (Viet-U), whileXLSR-53-Vietused the multilingualXLSR-53model as initialization and was then pre-trained onViet-U. - Fine-tuning: These pre-trained models were then fine-tuned on the labeled medical data (
VietMed-Train). They also experimented with language models built using different lexicons.
Datasets
- Newly Introduced:
- VietMed: A comprehensive Vietnamese medical speech dataset comprising:
VietMed-L: 16 hours of labeled medical speech (split into train, dev, test).VietMed-U: 1000 hours of unlabeled medical speech.-
Viet-U: 1200 hours of unlabeled general-domain speech. This dataset is notable for its size, coverage of all ICD-10 disease groups, and inclusion of all Vietnamese accents. -
Existing Datasets Used for Pre-training/Language Modeling:
XLSR-53: A multilingual speech representation model.
- Various text corpora for language model training, including VIVOS, BABEL, CommonVoice, FOSD, VNTC-Health, VLSP 2020, ViHealthBERT-FAQ, and PhoNER-Covid19.
Key Insights and Ideas
- Data Scarcity is a Major Bottleneck: The paper strongly emphasizes how the lack of public medical speech data, especially for languages like Vietnamese, is a significant barrier to progress in medical ASR.
- Comprehensive Coverage Matters: By including all ICD-10 disease groups and all regional accents, VietMed aims to improve the generalizability of ASR models beyond narrow use cases.
- Pre-training is Key for Low-Resource Domains: Even without using any in-domain medical data during the unsupervised pre-training phase, models initialized with large pre-trained representations (like XLSR-53) and then pre-trained on general Vietnamese data (
Viet-U) show remarkable performance when fine-tuned on medical data. - Computer-Assisted Annotation is Efficient: The authors found that using a native speaker with a Biomedical Engineering degree to correct automatically generated transcripts reduced annotation time by 70% while improving quality.
Key Results
- VietMed is the Largest: The paper claims VietMed is the world's largest public medical speech recognition dataset across several metrics (duration, speakers, diseases, etc.) and the largest public Vietnamese speech dataset by duration.
- State-of-the-Art Performance: Their
XLSR-53-Vietmodel, fine-tuned onVietMed-Train, achieved a Word Error Rate (WER) of 29.6% on the test set. This is a significant improvement over the baselineXLSR-53model, which achieved 51.8% WER, representing a relative reduction of over 40%. - Competitive Models: Their
w2v2-Vietmodel, pre-trained from scratch on general Vietnamese data, was competitive withXLSR-53despite using significantly less data for pre-training. - Language Model Improvements: Augmenting the lexicon with their
VietMed-Traindata improved language model perplexity (PPL).
Limitations and Future Work
- Unsupervised Pre-training on In-Domain Data: The authors acknowledge that they did not adapt their unsupervised pre-training models (
w2v2-Viet,XLSR-53-Viet) using the unlabeled medical data (VietMed-U). They believe this could further enhance performance and leave it as future work. - Generalizability to Other Languages: While they aim to contribute to future research in medical ASR for other languages, the current work is focused on Vietnamese.
- Further Exploration of Annotation: While their computer-assisted approach was effective, further research into optimal annotation strategies for medical speech could be beneficial.
Summary updated