Health, security and fire safety process optimisation using intelligence at the edge

Ollencio D'Souza, Subhas Chandra Mukhopadhyay, Michael Sheng

TL;DR

This paper introduces a TinyML-based edge intelligence framework for real-time health, security, and fire safety monitoring. By fusing data from multiple sensors and processing it locally on low-power microcontrollers, the system aims to provide faster, more accurate, and energy-efficient event detection, crucial for life-saving applications.

What Problem Does This Paper Address?

The proliferation of IoT devices generates massive amounts of data, overwhelming traditional cloud-centric processing. This leads to increased latency, potential data loss, and challenges in maintaining temporal synchronization, especially critical for time-sensitive applications like health monitoring, security alerts, and fire safety. Existing systems often struggle with the sheer volume and complexity of sensor data, leading to delays in response and potential inaccuracies. The paper specifically addresses the need for efficient, real-time analysis of sensor data at the "edge" – closer to the data source – to overcome these limitations.

Methods

The core of this research lies in developing an "intelligent edge" system using TinyML models. Key techniques include:

  • Multisensor Fusion: Combining data from various sensors (e.g., motion, temperature, humidity, audio, visual) to create a more comprehensive understanding of events than any single sensor could provide. This is achieved through "collaborate, corroborate, and reinforce" techniques.
  • TinyML Models: Compressing larger machine learning models into lightweight versions suitable for resource-constrained microcontrollers (MCUs). This allows for on-device processing.
  • Domain-Specific "Wake Up" Triggers: Implementing low-power listening modes for MCUs, activated by specific "wake words" (like "Help Me!") or anomaly detection. This conserves energy by only activating full processing when necessary.
  • Edge Analytics: Performing data analysis and generating "actionable intelligence" directly on the edge device, reducing the need to transmit raw data to a central server.
  • Federated Learning (proposed): While not fully implemented in this study, the authors suggest federated learning as a scalable approach for future work, where models are trained across multiple edge devices without centralizing raw data.

Datasets

The paper describes data collection from two primary sources:

  • Simulated Data: Used when direct site access was not possible.
  • Anonymized Real-World Data: Provided by an external source, including:
  • Access control data (entry/egress events).
  • "Nurse call" data (health care resident activity).

The paper also mentions the use of a cloud service provider's interface for data upload and ML model creation, which likely utilizes its own internal datasets for training and validation.

Key Insights and Ideas

  • "Intelligence at the Edge" is Crucial for Real-Time Response: The paper strongly advocates for processing data locally on edge devices to achieve the necessary speed and temporal accuracy for critical applications.
  • Multisensor Fusion Enhances Accuracy and Reduces False Alarms: By combining data from diverse sensors, the system can more reliably detect events and distinguish them from noise or irrelevant signals, significantly reducing false alarms.
  • TinyML Enables Powerful AI on Low-Power Devices: The research demonstrates that sophisticated machine learning models can be effectively deployed on small, energy-constrained microcontrollers, opening up new possibilities for embedded intelligence.
  • "Wake Words" are Effective for Energy Conservation: Using specific audio triggers to activate processing is a practical strategy for managing power consumption in always-on monitoring systems.
  • Human Logic is the Foundation, ML Amplifies It: The ML models are trained on human-defined logic and examples, but ML's ability to find complex associations far surpasses human capabilities, leading to more robust and nuanced detection.

Key Results

  • "Wake Word" Accuracy: The "Help Me!" wake word achieved a 95.4% accuracy rate in simulations.
  • Event Detection Trials: Multiple trials for event detection showed varying performance, with accuracy reaching up to 71.43% in Trial-3. Precision, Recall, False-Positive Rate (FPR), False-Negative Rate (FNR), and Specificity (Sp) were also reported, demonstrating consistency across different detection outcomes.
  • Energy Efficiency: The system is designed to operate at very low power (less than 1 mW in listening mode, up to 10 mW during processing).
  • Reduced Payload: By sending only analyzed results ("actionable intelligence") rather than raw data, the system significantly reduces the data payload transmitted over networks.

Limitations and Future Work

  • Limitations:
    • The paper acknowledges that the "garbage in, garbage out" principle applies to ML; if the input data or reasoning is compromised, the outcomes can be magnified.
  • Some tests required simulated data due to site access limitations.
  • The performance of the ML models is highly dependent on the quality and quantity of training data.
  • Future Work:
    • Further research into energy harvesting and wireless power to extend system operational life.
  • Exploring federated learning for enhanced scalability and privacy.
  • Investigating more sophisticated customized sensor models and techniques.
  • Extending the application to other domains with similar detection requirements.
  • Developing more robust methods for handling data corruption and network constraints.

Summary updated

Abstract

The proliferation of sensors to capture parametric measures or event data over a myriad of networking topologies is growing exponentially to improve our daily lives. Large amounts of data must be shared on constrained network infrastructure, increasing delays and loss of valuable real-time information. Our research presents a solution for the health, security, safety, and fire domains to obtain temporally synchronous, credible and high-resolution data from sensors to maintain the temporal hierarchy of reported events. We developed a multisensor fusion framework with energy conservation via domain-specific “wake up” triggers that turn on low-power model-driven microcontrollers using machine learning (TinyML) models. We investigated optimisation techniques using anomaly detection modes to deliver real-time insights in demanding life-saving situations. Using energy-efficient methods to analyse sensor data at the point of creation, we facilitated a pathway to provide sensor customisation at the “edge”, where and when it is most needed. We present the application and generalised results in a real-life health care scenario and explain its application and benefits in other named researched domains.
BibTeX
@inproceedings{dsouza2022health,
  author = {Ollencio D'Souza and Subhas Chandra Mukhopadhyay and Michael Sheng},
  title = {{Health, security and fire safety process optimisation using intelligence at the edge}},
  booktitle = {Sensors},
  year = {2022},
  url = {https://openreview.net/forum?id=MgF57HrsNw},
}

Loading related content...

Figure