TL;DR
This paper introduces a novel system, PDATC, that uses a low-cost infrared thermal sensor to intelligently activate and deactivate the computationally intensive MediaPipe pose detection algorithm on embedded systems. This approach significantly reduces CPU workload and power consumption, especially in environments with low activity, by only running pose detection when a person is actually present.
What Problem Does This Paper Address?
Modern embedded systems are increasingly used for edge computing tasks like human pose detection. However, popular pose detection frameworks, such as MediaPipe, are notoriously CPU-intensive. They often run continuously, even when no person is in the scene, leading to wasted processing power, increased energy consumption, higher system temperatures, and unnecessary data generation. This is particularly problematic for resource-constrained embedded devices like single-board computers (SBCs) which have limited processing power and thermal dissipation capabilities. The paper aims to alleviate this by creating a more efficient pose detection system for embedded platforms.
Methods
The core of the proposed solution is the Pose Detection Aided by Thermal Camera (PDATC) system. PDATC employs a two-stage approach:
- Thermal Detection Algorithm (TDA): A low-cost, low-resolution infrared thermal camera (MLX90640BAA with 32x24 pixels) is used to detect potential human presence. The TDA processes raw thermal data to identify pixels within a human body temperature range. It then uses a Grassfire-based algorithm to group these pixels into "thermal areas of interest" (TAIs). Several filtering layers are applied, including removing small areas, checking for normal distribution of thermal data, and ensuring TAIs persist for a minimum duration. This algorithm is designed to be lightweight and procedural, minimizing CPU usage.
- Execution Manager (EM): This central component orchestrates the system. It initially only runs the TDA. If the TDA detects a potential TAI, the EM then activates MediaPipe's face detection algorithm (FDA) to confirm a human face within the TAI. If a face is confirmed and intersects with a TAI, the EM finally activates MediaPipe's pose detection algorithm (PDA) to track the person's pose. Crucially, the EM deactivates the FDA and PDA when no TAI is detected or when a TAI is no longer valid, thereby saving computational resources.
The system was implemented in Python and tested on various SBCs, including Orange Pi 5, Raspberry Pi 4B, Jetson Nano, and a minicomputer with an Intel N5105.
Datasets
The paper does not explicitly mention training datasets for the TDA. The evaluation is performed using real-world scenarios captured by the visible light camera and the thermal camera in a controlled environment (e.g., a bedroom). The performance is measured by comparing the system's behavior with and without the PDATC's intelligent activation mechanism.
Key Insights and Ideas
- Thermal Sensing as a Smart Trigger: The most significant insight is the effective use of a low-resolution thermal camera not for detailed analysis, but as a highly efficient, low-power trigger for computationally expensive vision tasks. This decouples the continuous monitoring requirement from the high-demand processing.
- Procedural TDA for Efficiency: Designing the TDA as a series of procedural filters rather than a complex deep learning model is key to its low CPU overhead. This allows it to run continuously without significantly impacting system resources.
- Hybrid Approach for Robustness: Combining the thermal sensor's ability to detect presence (regardless of lighting) with MediaPipe's robust face and pose detection provides a more reliable system. The TDA acts as a first-pass filter, reducing false positives that might otherwise activate the more demanding algorithms.
- Significant Power and CPU Savings: The results demonstrate that by intelligently deactivating MediaPipe, substantial reductions in CPU utilization and power consumption are achievable, especially in scenarios with infrequent human presence.
Key Results
- CPU Utilization Reduction: The PDATC system achieved significant reductions in average CPU usage. For example, on the Orange Pi 5, CPU usage dropped from 32.0% (only MediaPipe) to 18.2% with PDATC. The Raspberry Pi 4B saw a reduction from 63.8% to 32.2%.
- Power Savings: Power consumption was also reduced, with savings ranging from 17.76% to 30.54% across tested SBCs. The Orange Pi 5 and 4 LTS, along with the N5105 Ubuntu, achieved around 30% power savings.
- Elimination of False Positives: In scenarios with no subject present, the PDATC system effectively prevented MediaPipe's pose detection from running, thus eliminating false positives and the associated wasted resources.
- Temperature Reduction: Correspondingly, the CPU temperature of the SBCs decreased significantly when using PDATC, contributing to better system stability and potentially longer hardware lifespan. For instance, the Orange Pi 5's average body temperature dropped from 41.1°C to 36.3°C.
- FPS Performance: While the thermal camera operated at 8 FPS, and face detection was capped at 15 FPS, the pose detection algorithm's FPS varied by SBC. The system's overall responsiveness was maintained by only activating these components when necessary.
Limitations and Future Work
- Thermal Camera Resolution: The low resolution of the thermal camera limits its ability to discern fine details, necessitating the use of MediaPipe's face detection as a secondary confirmation step.
- False Positives in TDA: While the TDA is designed to minimize false positives, it's acknowledged that complex scenarios or unusual heat signatures could still lead to false positives, requiring the FDA to filter them. The paper mentions that the TDA needs further testing in more varied scenarios.
- Jetson Nano Limitations: The Jetson Nano's performance was somewhat hampered by software limitations and its EOL status, despite having an embedded GPU.
- Future Work: The authors suggest developing an enhanced TDA using a low-complexity neural network to further improve accuracy and potentially reduce reliance on the FDA. They also note that the thermal camera can be used to trigger other tasks, such as activating lights or uploading data, leading to more even CPU utilization. The hybrid sensor approach is also highlighted as a promising direction for future computer vision applications.
Summary updated