Mandarin Chinese Tone Recognition with an Artificial Neural Network

Li Xu, Wenle Zhang, Ning Zhou, Chao‐Yang Lee, Yongxin Li, Xiuwu Chen, Xiaoyan ZhaoJournal of Otology 2006 · DOI

TL;DR

This paper explores using artificial neural networks (ANNs) to automatically recognize the four tones in Mandarin Chinese. The study demonstrates that a simple feed-forward ANN can achieve high accuracy (around 90%) in classifying these tones, suggesting its potential as an objective tool for assessing tone production in children with cochlear implants.

What Problem Does This Paper Address?

Mandarin Chinese is a tonal language where the pitch contour of a syllable distinguishes word meaning. For individuals with cochlear implants, particularly those who are prelingually deaf, perceiving and producing these tones can be challenging. While cochlear implants provide some auditory information, pitch perception is often limited, and existing methods for assessing tone production in these children are not objective. This paper aims to evaluate the efficacy of artificial neural networks as a tool to objectively assess Mandarin Chinese tone production, which is crucial for effective communication and rehabilitation in this population.

Methods

The researchers employed a feed-forward backpropagation artificial neural network. The fundamental frequency (F0) contours of Mandarin Chinese monosyllabic words were extracted using an autocorrelation method and served as the input to the network. The network's architecture was varied, with the number of input features (derived from segments of the F0 contour) ranging from 1 to 16, and the number of neurons in the hidden layer also varying from 1 to 16. The output layer consisted of four neurons, each representing one of the four Mandarin tones. The network was trained using the Levenberg-Marquardt optimization algorithm.

Datasets

The study utilized speech samples from two groups:

  • Adults: 15 normal-hearing native Mandarin Chinese speakers (22-35 years old). Speech samples consisted of spontaneous productions of four Mandarin tone patterns across nine syllables, resulting in 540 tokens.
  • Children: 12 normal-hearing native Mandarin Chinese speaking children (3-6 years old). Speech samples were elicited by asking children to repeat the four tone patterns of 40 different Mandarin syllables, resulting in 1920 tokens.

The data was split into training and testing sets, with two-thirds used for training and one-third for testing.

Key Insights and Ideas

  • ANNs as a viable tool for tone recognition: The study confirms that ANNs are effective in recognizing Mandarin Chinese tones, even with relatively simple architectures.
  • Minimal input requirements: Surprisingly, the ANN could achieve high accuracy with a small number of input features (as few as 2) and hidden neurons (as few as 3 or 4) for adult speech samples.
  • Potential for objective assessment: The high accuracy achieved suggests that ANNs can offer an objective and efficient method for evaluating tone production, particularly in populations with hearing impairments.
  • Children's data presents a greater challenge: While ANNs performed well on adult data, achieving slightly lower but still impressive results on children's data, it highlights potential differences in tone production mastery between age groups.

Key Results

The artificial neural network was able to successfully classify Mandarin Chinese tone patterns with an accuracy of approximately 90% correct for both adult and children speech samples.

  • For adult speech samples, the network achieved over 90% accuracy with as few as 2 inputs and 3 or 4 hidden neurons.
  • For children's speech samples, the performance was slightly lower, just below 90% correct, and required a slightly greater number of input and hidden neurons to reach asymptotic performance.
  • When the number of input neurons or hidden neurons was as low as 1, the recognition performance dropped to chance levels (around 18-36% correct).

Limitations and Future Work

  • Limitations:
    • The study used normal-hearing children and adults. The performance of the ANN on speech from children with cochlear implants, who may have different tone production characteristics, remains to be tested.
  • The number of input features and hidden neurons required for optimal performance was slightly higher for children's speech samples compared to adults, suggesting potential differences in the complexity of their tone production.
  • The study focused on monosyllabic words. Tone recognition in continuous speech might present additional challenges.
  • Future Work:
    • The authors plan to train the neural network with speech samples from normal-hearing Mandarin-speaking children and then test it with speech samples from children with cochlear implants to assess its utility for evaluating tone production in this specific population.
  • Investigating the impact of different transfer functions (e.g., linear, radial basis functions) in the hidden-layer neurons to identify the optimal ANN architecture for tone pattern recognition.
  • Exploring the developmental aspects of tone production in children.
  • Examining the ANN's ability to recognize tones in other tonal languages like Cantonese and Thai, which have more than four tones.

Summary updated

Abstract

Mandarin Chinese tone patterns vary in one of the four ways, i.e, (1) high level; (2) rising; (3) low falling and rising; and (4) high falling. The present study is to examine the efficacy of an artificial neural network in recognizing these tone patterns. Speech data were recorded from 12 children (3-6 years of age) and 15 adults. All subjects were native Mandarin Chinese speakers. The fundamental frequencies (F0) of each monosyllabic word of the speech data were extracted with an autocorrelation method. The pitch data(i.e., the F0 contours) were the inputs to a feed-forward backpropagation artificial neural network. The number of inputs to the neural network varied from 1 to 16 and the hidden layer of the network contained neurons that varied from 1 to 16 in number. The output of the network consisted of four neurons representing the four tone patterns of Mandarin Chinese. After being trained with the Levenberg-Marquardt optimization, the neural network was able to successfully classify the tone patterns with an accuracy of about 90% correct for speech samples from both adults and children. The artificial neural network may provide an objective and effective way of assessing tone production in prelingually-deafened children who have received cochlear implants.
BibTeX
@article{xu2006mandarin,
  author = {Li Xu and Wenle Zhang and Ning Zhou and Chao‐Yang Lee and Yongxin Li and Xiuwu Chen and Xiaoyan Zhao},
  title = {{Mandarin Chinese Tone Recognition with an Artificial Neural Network}},
  journal = {Journal of Otology},
  year = {2006},
  doi = {10.1016/s1672-2930(06)50005-4},
}
Figure