Intelligent Microsystem for Sound Event Recognition in Edge Computing Using End-to-End Mesh Networking

Lulu Hou, Wenrui Duan, Guozhe Xuan, Shanpeng Xiao, Yuan Li, Yizheng Li, Jiahao ZhaoSensors 2023 · DOI

TL;DR

This paper introduces a compact, low-power intelligent microsystem capable of recognizing sound events (like alarms and water leaks) directly on the device using an efficient CNN. These microsystems form a flexible, smartphone-managed mesh network using BLE Mesh, enabling widespread, real-time environmental monitoring for applications like smart homes and security.

What Problem Does This Paper Address?

The Internet of Things (IoT) relies heavily on interconnected devices that can autonomously collect and process information. However, creating small, low-power "intelligent microsystems" that can perform complex tasks like sound event recognition at the "edge" (i.e., on the device itself) and communicate effectively in a network presents significant challenges. Existing solutions often suffer from large size, high power consumption, limited processing capabilities, or poor integration of sensing, processing, and networking. This leads to increased communication costs, response times, and a lack of scalability. The authors aim to bridge this gap by developing a highly integrated microsystem that balances miniaturization, low power, on-device sound recognition, and robust wireless mesh networking.

Methods

The core of this work is an intelligent microsystem designed with four key modules: sensing, processing, wireless transceiver, and power supply.

  • Hardware: A compact PCB integrates a low-power STM32WB55 MCU (with dual ARM Cortex-M4 and M0 cores for processing and Bluetooth), a high-performance ICS-43434 MEMS microphone, an antenna, and an energy management circuit powered by a lithium battery. The entire unit is designed for miniaturization, resulting in a volume of 7.36 cm³ and a weight of 8g (without housing).
  • Sound Event Recognition (SER): A Convolutional Neural Network (CNN) is employed for SER. To make it suitable for the resource-constrained microsystem, a lightweight CNN architecture was designed and trained. Crucially, the trained floating-point model was then quantized to an 8-bit fixed-point format using post-training quantization (PTQ). This significantly reduces model size, memory usage, and computational complexity, enabling it to run on the MCU's limited resources.
  • Wireless Networking: The microsystems utilize Bluetooth Low Energy (BLE) Mesh networking. This technology allows for a self-organizing, reliable, and low-power mesh network where devices can communicate through multiple "hops." Each microsystem acts as both a relay and a proxy node, enabling seamless integration with smartphones for monitoring and control.

Datasets

  • Training and Testing: The authors utilized two primary sources for data:
  • ESC-50 Dataset: A public dataset containing 2000 environmental recordings of 50 sound types. Specific sounds like alarms and door knocking were extracted.
  • Indoor Experimental Dataset: This dataset, captured using smartphones, included recordings of water flow sounds and indoor background noise.
  • Data Preprocessing: All audio data was resampled to 16 kHz, segmented into 1-second clips, and converted to a single channel. Data augmentation (altering timing and pitch) was applied to address the limited availability of certain sound samples.
  • Test Dataset for Networking: A separate test dataset was constructed, comprising audio samples of alarm and water flow sounds not used during model training, to evaluate the generalization capabilities of the networked system.

Key Insights and Ideas

  • Edge AI for Sound: The successful deployment of a CNN for sound event recognition directly on a tiny, low-power microsystem is a significant step towards truly intelligent edge devices. This avoids the latency and bandwidth issues of sending raw audio to the cloud.
  • BLE Mesh for Scalability: Using BLE Mesh for networking provides a flexible and scalable solution. The ability to manage the network via smartphone and the multi-hop communication capability are key to its practical application in diverse environments.
  • Quantization is Crucial: The paper highlights the critical role of model quantization (converting a floating-point model to fixed-point) in making complex AI models feasible on resource-constrained embedded systems. The significant reduction in RAM and flash usage without a substantial drop in accuracy is a major takeaway.
  • Integrated Design: The holistic approach, optimizing hardware (size, power) and software (efficient CNN, BLE Mesh) in tandem, is essential for achieving the desired performance.

Key Results

  • Microsystem Size and Power: The microsystem measures 7.36 cm³ and weighs 8g (without housing). In sleep mode, it consumes only 0.18 mW, increasing to 36 mW in active mode.
  • Sound Event Recognition Accuracy:
    • Trained Samples: Achieved 100% accuracy for alarm sounds >70 dB and water flow sounds >50 dB within 6m.
  • Untrained Samples (within 3m): Achieved 96.30% accuracy for alarm sounds >70 dB and 92.50% for water flow sounds >55 dB.
  • Networking Performance:
    • Direct communication range between smartphone and microsystem: ~10 m.
  • Direct communication range between microsystems: ~5 m.
  • Model Compression: Quantization reduced peak RAM usage by 71.94% and peak flash usage by 73.98% with only a 0.25% drop in accuracy.

Limitations and Future Work

  • Limited Sound Types: The current system is trained to recognize specific sound events (alarms and water flow) and a limited set of noise types. Expanding the repertoire of recognizable sounds would increase its applicability.
  • Environmental Factors: While tested indoors, performance in highly noisy or complex acoustic environments might require further investigation and model refinement.
  • Untrained Data Performance: While good, the accuracy on untrained samples is slightly lower than on trained ones, suggesting potential for further improvement in generalization.
  • Future Work: The authors suggest that this microsystem can serve as the sensing layer for IoT applications in smart homes and unattended surveillance, implying future work could focus on integrating it into larger, more complex systems and exploring additional sensing modalities.

Summary updated

BibTeX
@article{hou2023intelligent,
  author = {Lulu Hou and Wenrui Duan and Guozhe Xuan and Shanpeng Xiao and Yuan Li and Yizheng Li and Jiahao Zhao},
  title = {{Intelligent Microsystem for Sound Event Recognition in Edge Computing Using End-to-End Mesh Networking}},
  journal = {Sensors},
  year = {2023},
  doi = {10.3390/s23073630},
}
Figure