TL;DR
This paper introduces MPSN, a novel pseudo-Siamese network that leverages motion information from video frames to improve indoor head detection. By focusing on the dynamic cues of moving heads and suppressing static backgrounds, MPSN achieves superior accuracy in challenging indoor environments, making it a promising tool for applications like building occupancy monitoring.
What Problem Does This Paper Address?
Accurate indoor occupancy detection is crucial for energy efficiency and comfort in buildings. While vision-based methods are common, detecting people in cluttered indoor scenes is challenging due to small object scales, diverse poses, and similar-looking background objects. Traditional head detection methods, often trained on static images, struggle with the dynamic nature of video, leading to high false positive and false negative rates. This paper addresses the need for a robust head detection system specifically designed for indoor video surveillance that can effectively distinguish moving heads from static clutter.
Methods
The core of MPSN is a Motion-aware Pseudo-Siamese Network. Here's a breakdown of its key components:
- Motion Estimation: The system first estimates motion information by calculating the pixel-wise difference between adjacent frames. This simple yet effective approach highlights moving objects, even small ones, and helps to filter out static background elements. They also explore using optical flow, but find frame difference to be more computationally efficient and competitive.
- Pseudo-Siamese Network: Unlike traditional Siamese networks that share weights, MPSN uses a pseudo-Siamese architecture. This allows it to process the original frame and the motion-derived frame difference independently with separate, but identically structured, networks (FN/FDN and BN/BDN). This is crucial because the motion information is fundamentally different from the original image data.
- Feature Aggregation (SFA & DFA):
- Shallow Feature Aggregation (SFA): At lower layers, MPSN simply sums the features from the original frame and the motion difference frame. This enhances motion regions.
- Deep Feature Aggregation (DFA): At deeper layers, MPSN employs a more sophisticated aggregation. It uses the motion features (specifically, the sigmoid-activated features from the motion branch) as a "mask" to selectively activate regions in the original frame's deep features. This helps to suppress static background features while preserving and enhancing those corresponding to moving heads.
- Detection Network: Finally, a standard detection network (like RPN) is used to predict bounding boxes and class probabilities from the aggregated features.
Datasets
The paper evaluates MPSN on two publicly available indoor video datasets:
- Brainwash dataset: A crowd dataset with 11,917 images and 91,146 labeled people.
- Restaurant dataset: Collected from four indoor restaurant locations, containing 1,610 images.
Both datasets are noted for their challenging indoor scenes and the presence of static objects that can be confused with heads.
Key Insights and Ideas
- Motion as a Discriminator: The fundamental insight is that human heads in indoor surveillance are often moving, while most background objects are static. Exploiting this difference through motion cues is a powerful way to improve detection accuracy.
- Pseudo-Siamese for Heterogeneous Inputs: Recognizing that motion features and original image features are not directly comparable in a standard Siamese setup, the pseudo-Siamese approach allows for tailored processing of each input type before aggregation.
- DFA as a "Learned Mask": The Deep Feature Aggregation (DFA) acts like a learned, dense mask that intelligently highlights relevant features from the original image based on motion cues, effectively suppressing noise from static objects.
- Flexibility of the Approach: The SFA and DFA modules can be integrated into various existing CNN architectures, demonstrating the generalizability and flexibility of the proposed motion-aware feature aggregation.
Key Results
- Superior Accuracy: MPSN consistently outperforms existing state-of-the-art methods on both the Brainwash and Restaurant datasets, achieving higher AP50 scores across different backbone networks. For example, with the MobileNetv2 backbone on the Brainwash dataset, MPSN achieved an AP50 of 0.90, significantly better than other methods.
- Improved Robustness: MPSN shows better performance than baseline methods even under adversarial attacks, particularly for small perturbation levels.
- Efficiency: While achieving high accuracy, MPSN with lighter backbones like MobileNetv2 offers a good balance between computational complexity and performance, making it suitable for edge deployments.
- Occupancy Counting Application: When applied to occupancy counting, MPSN demonstrates good performance with low normalized mean absolute error (NMAE), indicating its practical utility.
Limitations and Future Work
- Constant Time Interval: The paper notes that the current MPSN assumes a constant time interval between frames for motion estimation. Future work could explore handling variable time intervals.
- Edge Deployment: While MPSN with MobileNetv2 is efficient, further optimization for edge devices like Raspberry Pi and Jetson Nano is a promising direction.
- Adversarial Robustness Threshold: The paper discusses the trade-off in adversarial robustness based on the perturbation level ($\epsilon$), suggesting that the choice of model might depend on the expected level of attack. Further research into mathematically selecting robust models for specific $\epsilon$ values is mentioned.
- Post-processing for Counting: For occupancy counting, MPSN is a head detector. The authors acknowledge that post-processing techniques are needed to further improve counting performance, which is beyond the scope of this paper.
Summary updated