Abstract
In the analysis of massive surveillance video data, target behavior recognition is a critical task. Current research often focuses on lightweighting convolution operators or increasing module complexity, but frequently overlooks point-by-point convolution modules that consume significant parameters and computation. This paper presents a lightweight framework for intelligent recognition of key frame target behavior. By extracting 3D bone joint positions and utilizing a Local Vector Aggregation Descriptor (LVAD), the system creates compact representations of surveillance key frames. Through structured filter pruning and over-parameterized convolutions, the model achieves substantial reductions in memory and computational overhead while maintaining high accuracy.
Introduction
The ubiquity of video surveillance has led to an explosion of data, making manual monitoring impractical. While deep learning offers automated solutions for detecting behaviors like falls or medical emergencies, these models are often too computationally intensive for real-time deployment on edge hardware. A significant bottleneck is the point-by-point convolution module, which often accounts for a large portion of a model's parameters. This research addresses this gap by developing a lightweight recognition system that optimizes these modules without sacrificing expressive power.
Methodology
3D Bone Joint Feature Extraction
The system moves beyond raw pixel processing by extracting 3D coordinates of bone joints. Two primary feature types are defined:
- Relative Position Vectors: Measuring the distance between different joints within a single frame.
- Relative Displacement Vectors: Capturing the temporal movement of specific joints across frames.
Local Vector Aggregation Descriptor (LVAD)
To represent video sequences efficiently, the authors developed the LVAD, inspired by Bag-of-Words and Fisher vectors. The process involves clustering local features using K-means to identify "codewords" for specific behaviors. The differences between actual features and these cluster centers are then aggregated into a fixed-size, highly informative descriptor.
Model Optimization: Pruning and Over-parameterization
The core of the lightweighting strategy involves two steps:
- Filter Structured Pruning: $L_1$ and $L_2$ regularization are applied to the scale factors of Batch Normalization layers to identify and remove redundant channels.
- Over-parameterized Convolutions: To mitigate accuracy loss from pruning, extra depth is introduced during training. These complex layers are mathematically collapsed into simpler, faster structures for inference.
Key Insights and Results
Feature Fusion and Performance
A critical finding of this study is the necessity of fusing spatial and temporal data. While relative position data alone yields approximately 60% accuracy, combining it with displacement data pushes accuracy above 80%. This underscores that the dynamics of movement are as vital as static posture for behavior recognition.
Efficiency Gains
The experimental results demonstrate significant "slimming" of standard architectures:
- VGG-19: Pruning 65% of the channels reduced the number of parameters by nearly 80% and floating-point operations (FLOPs) by 60%.
- DenseNet-121: This backbone achieved an accuracy of 96.35% after optimization, showing that pruning can sometimes improve generalization.
- Robustness: Models utilizing over-parameterized convolutions consistently outperformed standard models across various pruning rates, proving the effectiveness of the training approach.
Conclusion
This research successfully demonstrates a lightweight framework for behavior recognition in surveillance videos. By focusing on point-by-point convolution optimization and effective feature representation through LVAD, the proposed method achieves high accuracy with a fraction of the original model's size. Future work will explore multi-person scenarios and further refine the LVAD generation process for ultra-low-power edge devices.
Summary updated