Abstract
Recently, deep learning based video object detection has attracted more and more attention. Compared with object detection of static images, video object detection is more challenging due to the motion of objects, while providing rich temporal information. The RNN-based algorithm is an effective way to enhance detection performance in videos with temporal information. However, most studies in this area only focus on accuracy while ignoring the calculation cost and the number of parameters. In this paper, we propose an efficient method that combines channel-reduced convolutional GRU (Squeezed GRU), and Information Entropy map for video object detection (SGE-Net). The experimental results validate the accuracy improvement, computational savings of the Squeezed GRU, and superiority of the information entropy attention mechanism on the classification performance. The mAP has increased by 3.7 contrasted with the baseline, and the number of parameters has decreased from 6.33 million to 0.67 million compared with the standard GRU.
Introduction
Object detection in video sequences is significantly more complex than in static images. While videos provide rich temporal context, they also introduce challenges like motion blur, occlusion, and varying object appearances. Current state-of-the-art methods often use Recurrent Neural Networks (RNNs) like Gated Recurrent Units (GRUs) or Long Short-Term Memory (LSTM) networks to bridge the gap between frames. However, these modules are often computationally expensive and parameter-heavy, making them difficult to deploy in real-time scenarios like robotics or mobile surveillance. Furthermore, standard Convolutional Neural Networks (CNNs) suffer from limited receptive fields, often missing the global spatial context of a frame. SGE-Net addresses these issues by capturing both temporal correlations and global spatial cues without the massive computational overhead typical of recurrent architectures.
Methodology
The SGE-Net framework integrates two primary innovations: the Squeezed GRU and the 2D Information Entropy (IE) map. The overall architecture, shown below, uses a MobileNet-SSD backbone, inserts the IE map for feature enhancement, and places the Squeezed GRU at the end of the feature extraction phase to model temporal flow.
Squeezed GRU
Traditional GRUs treat images as vectors, losing spatial correlation. While Convolutional GRUs (ConvGRU) preserve spatial structure, they remain parameter-heavy. The Squeezed GRU optimizes this by replacing standard convolutions with depth-wise separable convolutions and implementing channel reduction (using $1 \times 1$ convolutions) to shrink the feature space before processing. This allows the model to maintain spatial-temporal correlations with only ~10% of the parameters of a standard GRU.
2D Information Entropy (IE) Map
Traditional entropy provides a single global value for an image, which lacks spatial detail. The authors propose a 2D IE map calculated via a sliding window that captures the relationship between a pixel's gray value and the average gray value of its neighbors. Because this is a deterministic calculation based on the input image, it acts as a "free" attention mechanism that helps the network locate and classify objects based on their unique entropy distributions without adding any learnable parameters.
Experimental Results
The model was evaluated on the ImageNet VID 2015 dataset, consisting of 3,862 training video snippets and 555 testing snippets across 30 object categories.
- Accuracy: SGE-Net achieved a 46.8% mAP, a significant improvement over the 43.1% baseline (MNet-SSD).
- Efficiency: The Squeezed GRU module contains only 0.67 million parameters, compared to 6.33 million in a standard GRU and 29.49 million in a ConvGRU.
- Computational Savings: Compared to a standard ConvGRU, SGE-Net saves over 40% in computational costs (measured in Giga-MACs).
- IE Map Impact: Adding the IE map alone provided a 1.1% mAP boost over the Squeezed GRU baseline with zero increase in the parameter count.
Conclusion
SGE-Net demonstrates that efficient video object detection is possible by combining lightweight temporal modeling with traditional image processing techniques. By aggressively reducing channels in the GRU and utilizing a "zero-cost" entropy-based attention mechanism, the framework achieves superior accuracy with a fraction of the parameters required by conventional RNN-based detectors. Future work will explore applying these modules to other video-based tasks like semantic segmentation and multi-person pose estimation.
Summary updated