Two-Stage Training for Chinese Dialect Recognition

Zongze Ren, Guofu Yang, Shugong XuInterspeech 2019

TL;DR

This paper introduces a novel two-stage training system for Chinese dialect recognition, achieving state-of-the-art results by first training an acoustic model to predict phoneme sequences and then using these intermediate features to classify dialects. This approach significantly outperforms baseline methods and even a more complex three-stage system, demonstrating its efficiency and effectiveness for this challenging task.

What Problem Does This Paper Address?

Accurately identifying spoken language is a well-studied problem, but distinguishing between closely related dialects of the same language, such as Chinese, presents a much greater challenge. Chinese dialects often share the same written characters and have similar pronunciations, making traditional language identification (LID) methods less effective. Existing LID systems often focus on end-to-end training without explicitly leveraging phonetic information, which is crucial for differentiating subtle variations in pronunciation. This paper tackles the problem of Chinese dialect recognition by proposing a more nuanced, multi-stage training approach that explicitly incorporates phonetic information.

Methods

The core of the proposed system is a two-stage training strategy.

  • Stage 1: Acoustic Model (AM) Training: An acoustic model is trained using a Connectionist Temporal Classification (CTC) loss function. This allows the model to learn to recognize phoneme sequences directly from audio without requiring precise frame-level phonetic alignments. The model architecture consists of a shallow ResNet14 (a modified ResNet-18 with fewer parameters) for feature extraction, followed by a 2-layer Bidirectional Long Short-Term Memory (BLSTM) network. The CTC loss is used here to align the predicted phoneme sequences with the ground truth.
  • Stage 2: Dialect Classification: The intermediate features extracted by the ResNet14 from the first stage are then fed into a second network, also a 2-layer BLSTM. This network is trained using a cross-entropy loss to classify the dialect category. This approach leverages the phonetic understanding gained in the first stage to improve dialect discrimination.

The authors also explored a three-stage system for comparison. In this system, the first stage trains an AM with CTC to generate phoneme temporal locations. The second stage then trains another AM (without RNN) using cross-entropy loss to predict phonetic labels for each frame, utilizing the precise locations from the first stage. Finally, the third stage uses intermediate features from the second stage to train an RNN for LID, also with cross-entropy loss.

Datasets

The experiments were conducted on a newly compiled Chinese dialect database containing 10 widespread Chinese dialects. Each dialect has 6 hours of audio data. The training set consists of 6000 audio files per dialect with variable lengths. The test set includes 500 audio files per dialect, divided into two categories: short utterances (≤3s) and long utterances (>3s). The database also provides phonetic sequence annotations for the corresponding text.

Key Insights and Ideas

  • Phonetic Information is Key: Explicitly modeling phonetic sequences, even without precise alignment, significantly boosts dialect recognition performance compared to end-to-end systems.
  • Two-Stage Efficiency: The two-stage system achieves superior performance and requires less training time than a more complex three-stage approach. This suggests that the intermediate features from the first stage are highly informative and a simpler second stage is sufficient for dialect classification.
  • Shallow ResNet14 is Sufficient: A modified, shallower ResNet14 architecture proves effective for extracting relevant acoustic features, demonstrating that deep networks are not always necessary for this task, leading to fewer parameters and faster training.
  • Step-by-Step Learning: Training stages sequentially, rather than jointly (multi-task learning), is crucial for preserving acoustic knowledge and achieving better performance when dealing with features at different levels of abstraction (phonetic vs. dialectal).

Key Results

The proposed two-stage system achieved a remarkable 89% accuracy on the Chinese Dialect Recognition Challenge, winning first place among 110 teams.

  • Overall Accuracy: The two-stage system achieved an overall accuracy of 88.88%, a significant improvement over the baseline RNN system (78.85%).
  • Short vs. Long Utterances: The two-stage system performed excellently on both short (87.72%) and long (90.04%) duration tasks, demonstrating robustness.
  • Comparison with Three-Stage: The two-stage system outperformed the three-stage system (87.24% overall accuracy), which was slightly worse but still significantly better than the baseline.
  • RNN Structure Impact: For the two-stage system, BLSTM generally outperformed BGRU, and increasing BLSTM layers did not yield significant improvements.

Limitations and Future Work

  • Dialect Overlap: The paper acknowledges that some dialects, like Minnan and Kekka, or Hebei and Shanxi, are difficult to distinguish due to regional similarities and shared linguistic features.
  • Future Work: The authors suggest that this two-stage training paradigm could be extended to other classification tasks where hierarchical feature learning is beneficial. Further exploration of different RNN structures and optimization strategies could also be pursued.

Summary updated

BibTeX
@inproceedings{ren2019two,
  author = {Zongze Ren and Guofu Yang and Shugong Xu},
  title = {{Two-Stage Training for Chinese Dialect Recognition}},
  booktitle = {Interspeech},
  year = {2019},
}
Figure