SGE net: Video object detection with squeezed GRU and information entropy map

Rui Su, Wenjing Huang, Haoyu Ma, Xiaowei Song, Jinglu Hu

Abstract

Recently, deep learning based video object detection has attracted more and more attention. Compared with object detection of static images, video object detection is more challenging due to the motion of objects, while providing rich temporal information. The RNN-based algorithm is an effective way to enhance detection performance in videos with temporal information. However, most studies in this area only focus on accuracy while ignoring the calculation cost and the number of parameters. In this paper, we propose an efficient method that combines channel-reduced convolutional GRU (Squeezed GRU), and Information Entropy map for video object detection (SGE-Net). The experimental results validate the accuracy improvement, computational savings of the Squeezed GRU, and superiority of the information entropy attention mechanism on the classification performance. The mAP has increased by 3.7 contrasted with the baseline, and the number of parameters has decreased from 6.33 million to 0.67 million compared with the standard GRU.

Introduction

Object detection in video sequences is significantly more complex than in static images. While videos provide rich temporal context, they also introduce challenges like motion blur, occlusion, and varying object appearances. Current state-of-the-art methods often use Recurrent Neural Networks (RNNs) like Gated Recurrent Units (GRUs) or Long Short-Term Memory (LSTM) networks to bridge the gap between frames. However, these modules are often computationally expensive and parameter-heavy, making them difficult to deploy in real-time scenarios like robotics or mobile surveillance. Furthermore, standard Convolutional Neural Networks (CNNs) suffer from limited receptive fields, often missing the global spatial context of a frame. SGE-Net addresses these issues by capturing both temporal correlations and global spatial cues without the massive computational overhead typical of recurrent architectures.

Methodology

The SGE-Net framework integrates two primary innovations: the Squeezed GRU and the 2D Information Entropy (IE) map. The overall architecture, shown below, uses a MobileNet-SSD backbone, inserts the IE map for feature enhancement, and places the Squeezed GRU at the end of the feature extraction phase to model temporal flow.

Overall architecture of SGE-Net showing the integration of the IE map and Squeezed GRU.

Squeezed GRU

Traditional GRUs treat images as vectors, losing spatial correlation. While Convolutional GRUs (ConvGRU) preserve spatial structure, they remain parameter-heavy. The Squeezed GRU optimizes this by replacing standard convolutions with depth-wise separable convolutions and implementing channel reduction (using $1 \times 1$ convolutions) to shrink the feature space before processing. This allows the model to maintain spatial-temporal correlations with only ~10% of the parameters of a standard GRU.

Detailed internal structure of the Squeezed GRU module.

2D Information Entropy (IE) Map

Traditional entropy provides a single global value for an image, which lacks spatial detail. The authors propose a 2D IE map calculated via a sliding window that captures the relationship between a pixel's gray value and the average gray value of its neighbors. Because this is a deterministic calculation based on the input image, it acts as a "free" attention mechanism that helps the network locate and classify objects based on their unique entropy distributions without adding any learnable parameters.

Experimental Results

The model was evaluated on the ImageNet VID 2015 dataset, consisting of 3,862 training video snippets and 555 testing snippets across 30 object categories.

  • Accuracy: SGE-Net achieved a 46.8% mAP, a significant improvement over the 43.1% baseline (MNet-SSD).
  • Efficiency: The Squeezed GRU module contains only 0.67 million parameters, compared to 6.33 million in a standard GRU and 29.49 million in a ConvGRU.
  • Computational Savings: Compared to a standard ConvGRU, SGE-Net saves over 40% in computational costs (measured in Giga-MACs).
  • IE Map Impact: Adding the IE map alone provided a 1.1% mAP boost over the Squeezed GRU baseline with zero increase in the parameter count.

Conclusion

SGE-Net demonstrates that efficient video object detection is possible by combining lightweight temporal modeling with traditional image processing techniques. By aggressively reducing channels in the GRU and utilizing a "zero-cost" entropy-based attention mechanism, the framework achieves superior accuracy with a fraction of the parameters required by conventional RNN-based detectors. Future work will explore applying these modules to other video-based tasks like semantic segmentation and multi-person pose estimation.

Summary updated

Abstract

Recently, deep learning based video object detection has attracted more and more attention. Compared with object detection of static images, video object detection is more challenging due to the motion of objects, while providing rich temporal information. The RNN-based algorithm is an effective way to enhance detection performance in videos with temporal information. However, most studies in this area only focus on accuracy while ignoring the calculation cost and the number of parameters. In this paper, we propose an efficient method that combines channel-reduced convolutional GRU (Squeezed GRU), and Information Entropy map for video object detection (SGE-Net). The experimental results validate the accuracy improvement, computational savings of the Squeezed GRU, and superiority of the information entropy attention mechanism on the classification performance. The mAP has increased by 3.7 contrasted with the baseline, and the number of parameters has decreased from 6.33 million to 0.67 million compared with the standard GRU.
BibTeX
@inproceedings{su2021sge,
  author = {Rui Su and Wenjing Huang and Haoyu Ma and Xiaowei Song and Jinglu Hu},
  title = {{SGE net: Video object detection with squeezed GRU and information entropy map}},
  booktitle = {ICIP},
  year = {2021},
  doi = {10.1109/icip42928.2021.9506081},
}
Figure