Abstract
Motion history image (MHI) and motion energy image (MEI) are effective for simple action recognition but lack velocity information. This paper proposes a method combining wavelet moments of these temporal motion descriptors with regional optical flow features to represent actions more comprehensively. Wavelet moments provide rotation, translation, and scale invariance, while regional optical flow captures velocity changes. Experiments on outdoor video sequences demonstrate the effectiveness of this approach in distinguishing actions where speed is a critical factor.
Introduction
Recognizing human behavior in computer vision often relies on temporal templates like Motion History Images (MHI) and Motion Energy Images (MEI). While these templates effectively capture the spatial extent and temporal progression of movement, they struggle to represent velocity. For instance, a "gentle touch" and a "fierce strike" may produce similar static motion templates despite being fundamentally different actions. Additionally, traditional descriptors like Hu Moments are sensitive to image noise. This research addresses these limitations by integrating wavelet-based shape descriptors with optical flow-based velocity features.
Methodology
The proposed framework utilizes a multi-stage pipeline for robust feature extraction:
- Foreground Extraction: A Mixture of Gaussians (MoG) model is employed for background subtraction, isolating the moving subject from environmental noise.
- Temporal Templates: MEI (a binary map of the total motion area) and MHI (a grayscale map where brightness represents recency) are generated to represent the action's "shape."
- Wavelet Moments: Cubic B-spline wavelet moments are applied to the MHI and MEI. Unlike global Hu moments, wavelets offer multi-resolution analysis, ensuring invariance to rotation, scale, and translation while maintaining robustness against noise.
- Regional Optical Flow: The Lucas-Kanade algorithm calculates optical flow specifically within the foreground regions to capture speed efficiently. Flow directions are binned into four 90° intervals and weighted by amplitude to create a velocity profile.
Experimental Setup
The method was evaluated using the Weizmann Database, which contains ten distinct actions: bending, jumping jacks, jumping, vertical jumping, running, sideways jumping, skipping, walking, waving one arm, and waving both arms. The dataset includes variations in subjects' physical characteristics and clothing to test the generalizability of the recognition system.
Results and Discussion
The proposed method (Wavelet + Optical Flow) achieved an average recognition rate of 85.5%, significantly outperforming the traditional Hu Moment approach (71.5%). Notably, the system achieved 100% accuracy for actions like Jumping Jacks, Vertical Jumping, Walking, and Waving one arm. The lowest performance was observed for "Skipping" (55.5%), likely due to its visual similarity to running and jumping in 2D projections. The inclusion of velocity features proved crucial for distinguishing actions with similar spatial patterns but different speeds.
Conclusion
By fusing multi-resolution wavelet moments with regional optical flow, this research provides a more descriptive and noise-resilient representation of human motion. While the current four-direction binning for optical flow is computationally efficient, future work could explore finer directional granularity and testing on more complex, cluttered real-world datasets to further enhance recognition accuracy.
Summary updated