Abstract
This paper presents a complete sound and speech recognition system designed for Health Smart Homes. By integrating environmental sound classification (AUDITHIS) with specialized speech recognition (RAPHAEL), the system monitors Activities of Daily Living (ADL) and detects distress situations for elderly residents, ensuring autonomy through non-intrusive audio monitoring.
Introduction
As the global population ages, maintaining independence for the elderly has become a critical societal challenge. Geriatricians often rely on the "Activities of Daily Living" (ADL) index to assess autonomy, but manual assessments are infrequent and subjective. While many smart homes utilize infrared (PIR) motion sensors or door contacts, these modalities lack context; they can identify a resident's location but cannot detect a verbal call for help or the specific nature of a fall.
Audio monitoring offers a rich data stream but faces significant technical hurdles in real-world environments, including high reverberation from glass surfaces, low Signal-to-Noise Ratios (SNR), and the acoustic variability of elderly voices. This research introduces an integrated multi-channel audio system designed to overcome these challenges.
System Architecture: AUDITHIS and RAPHAEL
The proposed architecture operates in real-time, managing eight microphone channels through a modular pipeline. The system is divided into two primary components: the AUDITHIS analysis engine and the RAPHAEL speech recognizer.
AUDITHIS (The "Ears")
AUDITHIS handles the initial processing of environmental sounds. Its pipeline includes:
- Noise Suppression: Utilizing the SPEEX library and echo cancellation, the system "subtracts" known audio sources (like a TV or radio) from the ambient room sound.
- Detection & Segmentation: An energy-based detector identifies sound events, which are then classified as either "speech" or "everyday life sound" using Gaussian Mixture Models (GMM).
- Sound Classification: Environmental sounds are analyzed using Linear-Frequency Cepstral Coefficients (LFCC). Unlike standard MFCCs used in speech recognition, LFCCs do not compress high-frequency information, which is vital for identifying sounds like breaking glass or falling objects.
RAPHAEL (The "Brain")
When AUDITHIS identifies a segment as speech, it is passed to RAPHAEL, an autonomous Automatic Speech Recognition (ASR) system. RAPHAEL uses a specialized vocabulary of 299 words focused on domotic commands (e.g., "turn on the light") and distress phrases (e.g., "Help!", "I fell"). This targeted approach increases robustness in noisy environments compared to general-purpose dictation systems.
Physical Implementation and Sensor Fusion
The system was validated in a real Health Smart Home (HIS) environment in Grenoble. To provide a holistic view of the resident's status, the audio system is integrated with a suite of other sensors.
As shown in the floor plan, eight microphones are strategically placed across the bedroom, living room, kitchen, bathroom, and hallway. This spatial distribution allows the system to track the resident's movement and capture audio events regardless of their location. By fusing audio data with PIR sensors and door contacts, the system can successfully identify seven distinct Activities of Daily Living.
Key Insights and Results
- LFCC Superiority: The choice of LFCC for environmental sounds proved critical, as it preserves the high-frequency components necessary to distinguish domestic sounds that MFCC would otherwise obscure.
- Reference Signal Cancellation: By treating the TV/Radio as a known reference signal rather than random noise, the system can effectively "ignore" entertainment media, allowing it to hear a resident's distress call even in a loud room.
- Real-World Performance: While the system achieved high accuracy in lab settings (27dB SNR), performance in the actual HIS dropped due to external noise (such as a nearby helipad) and internal reverberation. Despite this, the modular, multi-threaded architecture maintained real-time processing capabilities.
Conclusion and Future Work
The AUDITHIS and RAPHAEL systems demonstrate the viability of audio-based monitoring for elderly care. Future work will focus on adapting acoustic models specifically for the unique properties of elderly voices (pitch, jitter, and speech rate) and improving de-reverberation algorithms to handle modern apartment designs with extensive glass surfaces. Ultimately, the highest reliability in autonomy monitoring is achieved through the intelligent fusion of audio with traditional motion and contact sensors.
Summary updated