Abstract
Human Action Recognition (HAR) from depth video sequences often faces challenges such as redundant static information and the need for multi-scale feature representation. This paper presents a novel framework utilizing Laplacian Pyramid Depth Motion Images (LP-DMI) to extract discriminative motion features. By decomposing depth motion projections into multi-scale Laplacian pyramids, the method effectively isolates dynamic contours while suppressing static noise. Experimental results on MSRAction3D, UTD-MHAD, and DHA datasets demonstrate that this approach achieves state-of-the-art accuracy with high computational efficiency using Extreme Learning Machines (ELM).
Introduction
Depth sensors, such as the Microsoft Kinect, have significantly advanced HAR by providing 3D structural information and robustness against lighting variations. However, many existing algorithms operate at a single scale, failing to capture the nuances of actions that involve both large-scale body movements and fine-grained gestures. Furthermore, depth videos often contain significant "static noise"—portions of the body that remain stationary during an action—which can degrade recognition performance if not properly filtered.
To address these issues, we propose a multi-scale framework that focuses on dynamic boundaries. By leveraging Laplacian pyramids, the system highlights the sharp, moving edges of limbs while "zeroing out" non-moving regions like the torso, leading to a more robust representation of human motion.
Proposed Methodology
The proposed framework follows a structured pipeline designed to transform raw depth sequences into high-dimensional, multi-scale feature vectors for classification.
1. Depth Motion Image (DMI) Generation
The process begins by projecting the depth video sequence onto three orthogonal planes: front, side, and top. This creates a set of Depth Motion Images (DMIs) that encapsulate the 3D geometry of the action over time. To ensure consistency across different subjects and distances, these images undergo Region of Interest (ROI) normalization.
2. Multi-scale Laplacian Decomposition
To simulate multi-scale human perception, a Gaussian Pyramid is first applied to the DMIs. However, to specifically isolate motion, the framework constructs a Laplacian Pyramid. This is achieved by subtracting a blurred, downsampled version of the image from the original at each level. This high-pass filtering effectively removes the "flat" static areas of the body and emphasizes the dynamic boundaries of moving limbs.
3. Feature Extraction and Classification
Histograms of Oriented Gradients (HOG) are extracted from each level of the Laplacian pyramid (LP-DMI-HOG). This captures "multi-granularity" features, analyzing the action at various resolutions simultaneously. For classification, the framework employs an Extreme Learning Machine (ELM). ELMs are chosen for their superior training speed compared to traditional deep learning architectures, as they do not require iterative backpropagation.
Experimental Results
The LP-DMI method was evaluated against three major public benchmarks, consistently outperforming existing state-of-the-art handcrafted and deep learning methods:
- MSRAction3D: Achieved 93.41% accuracy, notably higher than DMM-HOG (88.7%) and the deep-learning-based HP-DMM-CNN (92.3%).
- UTD-MHAD: Achieved 85.12% accuracy.
- DHA Dataset: Achieved 91.94% accuracy.
Key findings indicate that the optimal number of pyramid layers is between 3 and 4. Beyond this, the images become too small, and redundant noise begins to degrade performance. The use of Principal Component Analysis (PCA) for dimensionality reduction further ensures that the system remains computationally efficient for real-time applications.
Conclusion
By focusing on the "Goldilocks zone" of multi-scale motion features and stripping away redundant static data, the LP-DMI framework provides a highly accurate and efficient solution for depth-based action recognition. Future work will explore multimodal fusion—combining depth features with skeletal or RGB data—to better distinguish between actions with similar geometric trajectories.
Summary updated