TL;DR
This paper introduces HITHOME07, a novel, continuous audio database recorded in realistic home environments over extended periods. The goal is to enable research into truly "always-on" speech interfaces by capturing not just commands, but also the vast majority of time when the interface is not in use, a critical challenge for automatic activation.
What Problem Does This Paper Address?
The dream of a seamless, always-listening speech interface for home appliances hinges on a crucial ability: knowing when to listen and, more importantly, when to sleep. Current speech interfaces often rely on explicit triggers (like button presses) or system prompts, which interrupt the natural flow of interaction. The real challenge lies in developing systems that can automatically detect when a user intends to speak a command, without being constantly activated by every utterance or background noise. This requires understanding the nuances of human speech and non-speech in everyday home settings, a gap that existing databases, often focused on specific tasks or short recordings, don't fully address.
Methods
The authors developed a voice-activated TV controller as a proxy for a pervasive speech interface. This controller, disguised to look like an infrared remote, was equipped with trigger buttons and three microphones. The core of their approach involved recording continuous audio in home environments for extended periods (morning to evening). To capture diverse acoustic conditions, they employed microphone arrays, including those on the controller itself and ceiling-mounted arrays. For evaluation, they implemented a speaker-independent, isolated word recognition system based on HMMs. They also explored two activation strategies: trigger-initiated (where a button press signals an impending command) and automatic activation (where the system infers intent from the audio stream). For automatic activation, they experimented with power-based, array-based (using direction of arrival), and F0-based (pitch detection) speech/non-speech detection algorithms.
Datasets
The paper introduces the HITHOME07 (HITachi HOME environment audio database 2007). This database was recorded in two distinct home environments in Tokyo: a downtown business district apartment and a suburban residential studio. It comprises continuous audio data from multiple daily sessions, totaling approximately 270 hours. The recording setup included seven microphones and the TV's output audio, resulting in an 8-channel dataset. The database is designed to capture all aspects of daily life, including periods of speech, non-speech, and background noise, with a focus on the time when the speech interface is not actively used.
Key Insights and Ideas
- The "Sleeping" Interface is Key: The vast majority of time in a home environment is not spent issuing voice commands. Understanding and effectively managing this "sleeping" state is paramount for a truly ubiquitous speech interface.
- Home Environments are Complex: The HITHOME07 database highlights the diverse acoustic landscape of homes, from conversations and TV audio to background activities like cooking and cleaning. This complexity poses a significant challenge for automatic speech detection.
- Trigger-Initiated is Easier, but Limited: While trigger-initiated activation shows promising results (e.g., 85% recognition rate for in-vocabulary utterances), it still requires user intervention and doesn't solve the automatic activation problem.
- Automatic Activation is Tough: Preliminary experiments with automatic activation reveal that current methods struggle to achieve satisfactory performance, indicating a need for more robust speech/non-speech detection algorithms.
- Microphone Arrays Offer Potential: Using microphone arrays (e.g., for direction of arrival estimation) shows some improvement in speech/non-speech detection, especially when combined with other techniques, but further research is needed.
Key Results
- Trigger-Initiated Activation: Achieved an 85.0% recognition rate for in-vocabulary (IV) utterances and a 92.1% speech/non-speech detection rate. The overall task completion rate was 86.4%.
- Automatic Activation: Preliminary experiments with power-based speech/non-speech detection showed a precision of around 60% at best. Array-based and F0-based algorithms showed some improvements, but the paper notes that performance is not yet satisfactory. For instance, one array-based approach achieved a recall of 71.4% at a precision of 20.0%.
Limitations and Future Work
The authors acknowledge that the automatic activation system's performance is not satisfactory, highlighting the need for more research into robust speech/non-speech detection algorithms. They suggest that the HITHOME07 database itself will be a valuable resource for advancing these model-based approaches. Future work will likely focus on improving the accuracy and reliability of automatic activation systems, leveraging the rich data provided by the new database to better understand and model the complexities of home acoustic environments. The data discrepancy check and labeling processes are also ongoing, suggesting further analysis will be possible.
Summary updated