Polyphone Disambiguation for Mandarin Chinese Using Conditional Neural Network with Multi-Level Embedding Features

Zexin Cai, Yaogen Yang, Chuxiong Zhang, Xiaoyi Qin, Ming LiInterspeech 2019

TL;DR

This paper introduces a novel conditional neural network architecture for disambiguating polyphonic characters in Mandarin Chinese, achieving state-of-the-art accuracy. By leveraging multi-level embedding features from both word and sentence contexts, the system effectively resolves ambiguities that are crucial for natural-sounding text-to-speech synthesis.

What Problem Does This Paper Address?

Mandarin Chinese, like many languages, has characters that can be pronounced in multiple ways depending on their context. These are known as "polyphonic characters." This ambiguity poses a significant challenge for text-to-speech (TTS) systems, as selecting the incorrect pronunciation can lead to unnatural or even unintelligible speech. While grapheme-to-phoneme (G2P) conversion is a standard step in TTS, it doesn't inherently handle polyphony. Therefore, a dedicated polyphone disambiguation system is needed to determine the correct pronunciation of these characters based on their surrounding text. This problem is akin to resolving homographs in English, but with the added complexity of Mandarin's tonal nature.

Methods

The proposed system employs a conditional neural network architecture. At its core is a bidirectional Long Short-Term Memory (BLSTM) encoder, which processes the input character sequence to generate a sentence-level contextual embedding. This sentence-level feature captures long-range dependencies within the text. Additionally, the system utilizes a pre-trained word-to-vector lookup table to obtain word-level embeddings for the context surrounding the polyphonic character.

The architecture then combines three types of features:

  1. Polyphonic Character Embedding: A direct embedding of the polyphonic character itself.
  2. Word-Level Conditional Feature: An embedding derived from the word containing the polyphonic character.
  3. Sentence-Level Conditional Feature: The contextual embedding generated by the BLSTM encoder.

These concatenated features are fed into a prediction network consisting of several fully-connected layers with ReLU activation, culminating in a softmax layer that outputs the probability distribution over possible pinyin pronunciations. The authors experimented with different combinations of these conditional features (word-level only, sentence-level only, and both) to evaluate their impact.

Datasets

The authors used a publicly available dataset from Beijing Data-Baker Science and Technology Ltd. This dataset contains 150 frequently used polyphonic characters and 151,585 corresponding sentences. The corpus was split into a training set (140,794 sentences) and an evaluation set (10,791 sentences). The evaluation set was carefully constructed to represent 7% of character-pinyin pairs, with specific attention to ensuring representation of less frequent pairs.

Key Insights and Ideas

  • Context is King: The paper strongly reinforces the idea that both word-level and sentence-level context are crucial for accurately disambiguating polyphonic characters in Mandarin. Neither alone is sufficient for optimal performance.
  • Multi-level Embeddings are Powerful: Combining different types of contextual embeddings (word and sentence) leads to a significant performance boost, suggesting that these different levels of context capture complementary information.
  • BLSTM as a Feature Extractor: The BLSTM, while often used for sequence generation, proves highly effective as a feature extractor for sentence-level context in this disambiguation task.

Key Results

The proposed system achieved an impressive overall accuracy of 94.69% on the evaluation set. This represents a significant improvement over previous state-of-the-art methods, outperforming the best baseline system from [15] by 2.73%. The individual contributions of word-level and sentence-level features were also highlighted, with both showing substantial improvements over a system with no contextual information. The combined system (System CWC) that utilized both word and sentence-level conditional features delivered the best performance.

Limitations and Future Work

The paper acknowledges that while their system achieves high accuracy, the problem of polyphone disambiguation is "still not solved today." The authors focused on a specific set of polyphonic characters and a particular dataset. Future work could involve expanding the system to handle a wider range of polyphonic characters, exploring different neural network architectures, or investigating more sophisticated methods for context encoding. The paper also mentions the use of a pre-trained word-to-vector lookup table, implying that the quality and coverage of this table could influence performance.

Summary updated

BibTeX
@inproceedings{cai2019polyphone,
  author = {Zexin Cai and Yaogen Yang and Chuxiong Zhang and Xiaoyi Qin and Ming Li},
  title = {{Polyphone Disambiguation for Mandarin Chinese Using Conditional Neural Network with Multi-Level Embedding Features}},
  booktitle = {Interspeech},
  year = {2019},
}
Figure