Mutual information analysis for feature and sensor subset selection in surface electromyography based speech recognition

Vivek Kumar Rangarajan Sridhar, Rohit Prasad, Prem NatarajanInterspeech 2010

TL;DR

This paper introduces a novel framework using mutual information (MI) to systematically select the most informative features and sensor subsets for surface electromyography (sEMG)-based speech recognition. By moving beyond computationally expensive full recognition experiments, they demonstrate that Mel-frequency cepstral coefficients (MFCCs) are superior for sEMG, and that a carefully chosen subset of sensors can achieve performance comparable to using all available sensors.

What Problem Does This Paper Address?

Automatic speech recognition (ASR) systems typically rely on acoustic signals, which are susceptible to noise and environmental variations. Surface electromyography (sEMG), which captures electrical activity from speech-producing muscles, offers a potentially more robust alternative or complement. However, effectively utilizing sEMG signals for ASR presents two major challenges: 1) identifying the most discriminative signal features (e.g., MFCCs, time-domain features) and 2) determining the optimal subset of sensors to place on the body to capture these signals. Previous work often evaluated features and sensor configurations through full-scale recognition experiments, which are computationally intensive and don't provide a clear understanding of why certain choices are better. This paper aims to address this by providing a systematic, information-theoretic approach to feature and sensor subset selection.

Methods

The core of this paper is the application of Mutual Information (MI) as a fidelity metric. MI quantifies the amount of information one random variable (e.g., a feature vector) contains about another (e.g., the spoken word class).

  1. Feature Selection: The authors first use MI to evaluate different feature representations (MFCCs, Time-Domain (TD), Short-Time Fourier Transform (STFT), and Wavelet Transform (WT)) extracted from sEMG signals. To make MI computation tractable, they employ a quantization-based approximation (QMI). This involves discretizing the feature space and estimating probability distributions, allowing for efficient comparison of feature informativeness.
  2. Sensor Subset Selection: Building on the feature analysis, they extend the MI framework to select optimal subsets of sEMG sensors. Recognizing that using all available sensors can lead to data sparsity, they systematically analyze combinations of sensors. To reduce the combinatorial explosion, they fix a few consistently high-performing sensors and explore subsets of the remaining ones.
  3. Validation: Crucially, the findings from the MI analysis are validated using Hidden Markov Model (HMM)-based recognition experiments. This step confirms whether the features and sensor subsets identified as optimal by MI actually lead to improved recognition accuracy.

Datasets

The study utilized sEMG data collected from 9 subjects (5 male, 4 female) who are native American English speakers. The data was recorded under two conditions:

  • Voiced: Subjects speak words audibly.
  • Mouthed: Subjects perform the muscular actions of speaking without vocalization.

Each subject read out 65 words, three times, in two separate sessions. User-generated triggers were used for segmentation. The training set comprised four instances of each word, and the test set consisted of two instances.

Key Insights and Ideas

  • MFCCs Shine for sEMG: Surprisingly, Mel-frequency cepstral coefficients (MFCCs), traditionally used for acoustic speech, proved to be the most discriminative features for sEMG signals, even though they aren't intuitively linked to muscle activity. This suggests that the non-linear warping in MFCCs might capture underlying articulatory dynamics effectively.
  • MI as a Powerful Pre-selector: Mutual Information analysis provides a computationally efficient way to rank features and sensor subsets before running expensive recognition experiments. This allows researchers to quickly identify promising candidates.
  • Sensor Subsets Can Be Sufficient: The study demonstrates that a carefully selected subset of sensors (e.g., 5 sensors) can achieve recognition performance very close to, and sometimes even better than, using a larger set of 11 sensors. This has practical implications for reducing data complexity and potentially sensor placement effort.
  • Speaker-Specific Articulation: The analysis hints that different speakers might rely on different combinations of facial and neck muscles. Understanding these individual patterns could further optimize sEMG-based ASR.

Key Results

  • Feature Ranking: For sEMG signals, MFCCs yielded the highest mutual information, followed by Wavelet Transform (WT), STFT, and Time-Domain (TD) features. Traditional ASR MFCCs (derived from audio) showed even higher discrimination.
  • Sensor Subset Performance: The top-ranked sensor subset (identified via MI) achieved a mean recognition accuracy of 80.6% using HMMs, compared to 79.6% when using all 11 sensors. This indicates that the MI-based selection effectively removed redundant sensors.
  • Consistency: The sensor subsets identified as optimal by MI analysis consistently performed well in the subsequent HMM recognition experiments, validating the MI framework.

Limitations and Future Work

  • Statistical Insignificance: The paper notes that the differences in MI between some sensor subsets were statistically insignificant at a 0.01 alpha level.
  • Combinatorial Complexity: While MI reduces the need for full experiments, sensor subset selection still involves analyzing many combinations, though less than brute-forcing all possibilities.
  • Future Work: The authors suggest exploring normalization and adaptation techniques for session-independent and speaker-independent sEMG-based recognition. They also believe their framework could be useful for literacy assessment and guidance.

Summary updated

BibTeX
@inproceedings{sridhar2010mutual,
  author = {Vivek Kumar Rangarajan Sridhar and Rohit Prasad and Prem Natarajan},
  title = {{Mutual information analysis for feature and sensor subset selection in surface electromyography based speech recognition}},
  booktitle = {Interspeech},
  year = {2010},
}
Figure