Abstract
Automatic Speech Recognition (ASR) systems for Mandarin often struggle with regional accents, such as the Shanghai (Wu) dialect. This paper presents a multi-pronged approach to improve ASR for Shanghai-accented Mandarin by combining automatic accent detection, formant-augmented acoustic features, and a model-selection framework. By leveraging phonetic shifts and advanced adaptation techniques like Maximum Likelihood Linear Regression (MLLR) and Maximum A Posteriori (MAP), the researchers achieved a significant reduction in Character Error Rate (CER), bringing the speaker-averaged CER down to 40.7%.
Introduction
Standard Mandarin (Putonghua) is the official spoken language of China, but it is spoken with vast regional variations. Current ASR systems trained on "standard" speech often see a dramatic performance drop when encountering speakers from dialectal regions like Shanghai.
The challenge is that "accent" isn't a binary toggle; it’s a spectrum. Previous work often treated all speakers from a region as a single class. This paper addresses the need to treat accentedness as a matter of degree and explores how to automatically detect that degree to choose the best acoustic model for a specific speaker.
The authors validated that younger, female, and more educated speakers in Shanghai tend to have more "standard" pronunciation, while older generations retain stronger accents (Figure 1). This confirms that ASR systems must be flexible enough to handle varying degrees of deviation.
Methods
The authors propose a pipeline that moves beyond simple speaker adaptation:
- Phoneme-Based Accent Detection: Instead of just using spectral features (like MFCCs), the authors leverage a known phonetic shift in Shanghai-accented Mandarin: the tendency to replace retroflex fricatives (e.g., sh, ch, zh) with alveolar equivalents (e.g., s, c, z). They calculate the ratios of these sounds from decoding lattices to classify speakers into "more standard" or "more accented" groups.
- Formant-Augmented Features: Standard MFCCs often miss subtle accent cues. By appending formant frequencies ($F_1-F_3$) and, more importantly, formant amplitudes ($\eta$), the researchers created acoustic models that better capture the nuances of accented speech.
- Advanced Adaptation (MAP/MLLR): The system combines MLLR and MAP adaptation. Specifically, "phone-specific" transforms (IF-60) are used rather than a single global transform, allowing the model to capture systematic variations associated with specific sounds.
- Model Selection: Using the accent detector, the system dynamically selects the most appropriate acoustic model (either a standard model or an "accent-favorable" model) during the decoding process.
Experimental Setup
The target data consisted of a spontaneous speech corpus of 100 speakers (50 male, 50 female) speaking Wu-accented Putonghua. Baseline training data included Mandarin Broadcast News (MBN), HUB5 telephone conversations, and HKUST conversational Putonghua. Evaluation was performed on 20 speakers reserved for testing.
Results
The proposed methods yielded substantial improvements in recognition accuracy. Combining phone-specific MLLR with MAP (IF-60) reduced the CER from a baseline of 61.0% to 43.7% on the MBN-trained model.
As shown in Figure 2, the integration of formant amplitudes ($WU_\eta$) and combined adaptation techniques further lowered the CER. The best-integrated system, utilizing speaker-dependent (SD) adaptation and model selection, brought the speaker-averaged CER down to 40.7%.
Furthermore, the lattice-based accent detection achieved 78% accuracy for identifying "more standard" speakers and 72% for "more accented" speakers when compared against human expert judgments. Implementing the model selection strategy provided an additional 1.0% to 1.4% absolute reduction in CER over state-of-the-art adaptation techniques.
Conclusion
This research demonstrates that treating accent as a spectrum and incorporating phonetic-specific features can significantly enhance ASR performance for regional dialects. Future work may explore "soft" model selection, where multiple models are weighted and integrated directly into the ASR engine, and the use of larger datasets to generalize across the diverse sub-variations of the Wu dialect group.
Summary updated