A natural in-car speech interface to internet services using hybrid ASR

Hansjörg Hofmann, Ute Ehrlich, Klaus Bader, Ilona Nothelfer, André BertonInterspeech 2012

TL;DR

This paper presents a prototype for a natural, in-car speech interface that allows drivers to access internet services like news and social media without manual interaction. It tackles the challenge of driver distraction by using a hybrid Automatic Speech Recognition (ASR) system that combines onboard and cloud-based recognizers for flexible and robust natural language understanding.

What Problem Does This Paper Address?

The increasing prevalence of smartphones and the desire to stay "always connected" pose a significant safety risk in vehicles. Manual interaction with smartphones while driving leads to distraction and accidents, with a substantial percentage of car crashes linked to cellphone use. While in-car systems offer some internet access, they often rely on rigid, command-based speech interfaces that can overwhelm users with the sheer number of commands required for numerous services. This paper addresses the need for a more intuitive and natural speech interface that minimizes driver distraction and allows seamless access to a wider range of internet services.

Methods

The core of the system is a hybrid Automatic Speech Recognition (ASR) architecture designed to handle natural language input. This hybrid approach leverages:

  • Onboard ASR: A VoConR 32001 system from Nuance is used for recognition within closed domains, employing constraint grammars for improved accuracy in specific tasks.
  • Cloud-based ASR: For more complex or open-ended tasks, the system offloads recognition to cloud services. Specifically, Nuance's Dragon NaturallySpeaking server is used for general transcription, and Google Voice Search is employed for keyword searches.

The system employs a natural language understanding (NLU) module that uses flexible grammars, including "garbage values" and "unknown models," to interpret user utterances. A key aspect of the NLU is its ability to re-call the ASR (specifically Google's for keyword searches) when an initial recognition misses a crucial keyword, ensuring the utterance can be fully interpreted.

A dialogue manager (DM) then processes the understood intent, interacts with internet services, and orchestrates the output. Speech generation uses Nuance's Vocalizer for Automotive TTS engine. The Human-Machine Interface (HMI) concept emphasizes minimizing distraction, with a highly restrictive graphical user interface (GUI) that only displays essential information, encouraging drivers to keep their eyes on the road.

Datasets

Not discussed in the paper. The paper focuses on the system architecture and HMI concept rather than specific training or evaluation datasets.

Key Insights and Ideas

  • Hybrid ASR for Flexibility: Combining onboard and cloud-based ASR is crucial for handling the diverse recognition needs of natural language interfaces for internet services. This allows for efficient processing of common tasks locally while leveraging powerful cloud resources for more complex ones.
  • Re-recognition for Robustness: The strategy of re-calling the ASR for keyword spotting when initial recognition fails is a clever way to improve robustness in natural language understanding, especially for open-ended queries.
  • Minimizing Visual Distraction: The restrictive GUI design is a critical HMI insight. Presenting only the most vital information and relying on audio output for details is paramount for driver safety.
  • User-Adaptive Dialogue Strategy: The system's dialogue control is designed to be flexible and adapt to user input rather than following a rigid, pre-defined flow, enhancing the naturalness of the interaction.

Key Results

The paper presents a functional prototype that allows users to access internet services like news retrieval and social media communication (Facebook, Email) using natural speech. The system successfully integrates onboard and cloud-based ASR and TTS technologies. While quantitative results are not detailed, the authors state the prototype "performs well." The primary qualitative finding is the successful implementation of a natural and intuitive speech interface that aims to reduce driver distraction.

Limitations and Future Work

  • Grammar Restrictions: The authors acknowledge that the current grammar is still restricted, limiting the full expressiveness of natural language.
  • Multimodal Integration: They are actively working on improving grammar flexibility and integrating multimodal interaction, suggesting future work will involve richer interaction modalities beyond just speech and minimal visuals.

Summary updated

BibTeX
@inproceedings{hofmann2012natural,
  author = {Hansjörg Hofmann and Ute Ehrlich and Klaus Bader and Ilona Nothelfer and André Berton},
  title = {{A natural in-car speech interface to internet services using hybrid ASR}},
  booktitle = {Interspeech},
  year = {2012},
}
Figure