Distant Speech Recognition for Home Automation: Preliminary Experimental Results in a Smart Home

François Portet, Michel Vacher2013

TL;DR

This paper tackles the challenge of robust distant speech recognition in smart homes, a crucial step for natural voice control. They introduce and evaluate novel approaches like the Driven Decoding Algorithm (DDA) and a two-level DDA, which significantly improve both speech recognition accuracy and the ability to detect specific commands, even in noisy, real-world environments.

What Problem Does This Paper Address?

The dream of a truly "smart home" hinges on intuitive and seamless interaction. While voice control offers a natural interface, achieving reliable speech recognition in a home environment is fraught with difficulties. Unlike controlled lab settings with close-talking microphones, smart homes present a "distant speech" scenario. This means dealing with:

  • Background Noise: Appliances, TV, other people talking – the home is a noisy place.
  • Reverberation: Sound bouncing off walls can distort speech.
  • Varied Speaker Positions: Users aren't always directly in front of a microphone.
  • Uncontrolled Speech: People don't speak clearly or deliberately when giving commands in their own homes.

These factors drastically degrade the performance of standard Automatic Speech Recognition (ASR) systems, making them unreliable for critical tasks like controlling home automation or responding to distress calls. This paper aims to bridge this gap by developing and evaluating ASR techniques specifically tailored for these challenging smart home conditions.

Methods

The researchers explored several strategies to improve ASR performance in their smart home setup:

  • ASR System Comparison: They evaluated two established ASR toolkits: Sphinx and Speeral, using both generic and specialized language models.
  • Acoustic Model Adaptation: Techniques like Maximum a Posteriori (MAP) and Maximum Likelihood Linear Regression (MLLR) were used to adapt acoustic models to individual speakers, aiming to improve robustness.
  • Language Model Interpolation: To balance the specificity of domotic commands with the flexibility needed for general conversation, they interpolated generic large-vocabulary language models with specialized models trained on expected commands.
  • Multi-Source ASR (ROVER): They leveraged the multiple microphones in the smart home by combining the outputs of different ASR streams using the ROVER (Recognizer Output Voting Error Reduction) algorithm. This aims to capitalize on the complementarity of different microphone placements.
  • Driven Decoding Algorithm (DDA): This is a core novel contribution. DDA uses the output of one ASR stream (or a pre-defined set of sentences) to "drive" the decoding process of another stream. This allows for more efficient combination of information from multiple microphones and can leverage prior knowledge about expected utterances.
  • Two-Level DDA: A further refinement of DDA, this approach uses the first pass of decoding to identify potential sentences (e.g., domotic commands) and then uses these "spotted" sentences to guide the second decoding pass. This is particularly effective for sentence spotting tasks.

Datasets

The study utilized a newly introduced benchmark: the SWEET-HOME speech corpus. This corpus was collected in a real smart home environment (the DOMUS flat) equipped with 7 wireless microphones. It contains:

  • Phase 1: Spontaneous speech from participants performing daily activities, including casual sentences and phone conversations.
  • Phase 2: Read speech, specifically including domotic orders and distress sentences, recorded under various noise conditions (though only clean conditions were used for the reported results).

This dataset is notable for capturing realistic distant speech in an uncontrolled, multi-microphone environment, making it a valuable resource for evaluating smart home ASR systems.

Key Insights and Ideas

  • MLLR is Key for Adaptation: While MAP adaptation showed limited benefit, MLLR proved to be a crucial technique for adapting acoustic models to individual speakers, significantly reducing Word Error Rate (WER).
  • Specialized Models Need a Safety Net: Using only specialized language models for domotic commands led to good performance when users stuck to the script, but failed when they deviated. Interpolating with a generic language model provided the best of both worlds, offering robustness to unexpected utterances.
  • Multi-Microphone Fusion is Powerful: Combining information from multiple microphones, even with simple methods like ROVER, yielded substantial improvements over single-stream ASR.
  • DDA Offers Efficient Combination: The Driven Decoding Algorithm (DDA) provides a more computationally efficient way to combine information from multiple streams compared to ROVER, especially when confidence scores are not readily available.
  • Prior Knowledge is Gold: The two-level DDA, which leverages prior knowledge of expected sentences to guide the decoding process, demonstrated the most significant improvements, particularly for the critical task of detecting specific commands.

Key Results

  • Baseline Performance: Without adaptation, ASR systems achieved high WERs (e.g., 35% for Speeral).
  • MLLR Adaptation: With MLLR, Speeral's WER dropped to 14.5%.
  • ROVER Combination: Combining multiple systems with ROVER achieved a mean WER of 10.2%.
  • Driven Decoding (DDA):
    • The two-stream DDA achieved a mean WER of 12.3%.
  • The two-level DDA (driving with spotted sentences) achieved the best results, with a mean WER of 7.9% and a mean CER of 7.5%.
  • Sentence Spotting: The two-level DDA achieved 92.5% correct detection of predefined sentences, outperforming the baseline (83.1%) and other configurations.

Limitations and Future Work

  • Noise Robustness: While the study focused on distant speech, the paper acknowledges that unknown audio sources (like vacuums or fans) can still perturb the voice. Future work will involve exploring techniques like Independent Component Analysis (ICA) to separate speech from such undetermined noise.
  • Computational Resources: The paper notes that while ROVER is effective, it increases computation time. DDA offers a more efficient alternative, and future work will continue to explore low-cost, real-time solutions.
  • Acoustic Model Training Data: The paper mentions that differences in acoustic model training data between Sphinx and Speeral might influence performance comparisons and warrant further investigation.
  • Real-time Performance: While some DDA configurations achieved near real-time performance (1xRT), further optimization for truly real-time operation in a smart home context is an ongoing goal.

Summary updated

Abstract

This paper presents a study that is part of the Sweet-Home project which aims at developing a new home automation system based on voice command. The study focused on two tasks: distant speech recognition and sentence spotting (e.g., recognition of domotic orders). Regarding the first task, different combinations of ASR systems, language and acoustic models were tested. Fusion of ASR outputs by consensus and with a triggered language model (using a priori knowledge) were investigated. For the sentence spotting task, an algorithm based on distance evaluation between the current ASR hypotheses and the predefine set of keyword patterns was introduced in order to retrieve the correct sentences in spite of the ASR errors. The techniques were assessed on real daily living data collected in a 4-room smart home that was fully equipped with standard tactile commands and with 7 wireless microphones set in the ceiling. Thanks to Driven Decoding Algorithm techniques, a classical ASR system reached 7.9 % WER against 35 % WER in standard configuration and 15 % with MLLR adaptation only. The best keyword pattern classification result obtained in distant speech conditions was 7.5 % CER.
BibTeX
@misc{portet2013distant,
  author = {François Portet and Michel Vacher and François Portet},
  title = {{Distant Speech Recognition for Home Automation: Preliminary Experimental Results in a Smart Home}},
  year = {2013},
}
Figure