TL;DR
This paper tackles the challenge of deploying computationally intensive two-stage object detection models on resource-constrained devices. They introduce an "Efficient Feature Fusion Module" (EFFM) and a "Multi-Scale Dilation Region Proposal Network" (RPN) to significantly reduce operations and parameters in Faster R-CNN, achieving comparable accuracy with a fraction of the computational cost and model size.
What Problem Does This Paper Address?
Object detection, a crucial task in computer vision, has seen remarkable progress with deep convolutional neural networks (CNNs). However, these powerful models are often computationally demanding and have a large number of parameters, making them unsuitable for deployment on embedded systems or devices with limited computational power. Two-stage detectors, like Faster R-CNN, are particularly prone to this issue, with a significant portion of their computational burden and parameter count concentrated in the feature fusion stage after region proposal. This paper aims to bridge this gap by developing a more efficient two-stage object detection framework.
Methods
The authors propose two key innovations to enhance the efficiency of two-stage object detectors, specifically building upon the Faster R-CNN architecture:
- Efficient Feature Fusion Module (EFFM): This module is designed to reduce the computational cost and parameter count in the feature fusion stage. It employs:
- Pointwise convolution: Used for initial channel fusion and dimensionality reduction, which is more efficient than larger convolution kernels.
- Grouped convolution: Applied to extract spatial information, dividing the convolution into smaller groups to reduce operations.
- Grouped fully connected layers: Used to merge features, further reducing parameters by dividing the large fully connected layers into smaller, independent groups. This approach aims to maintain dense matrix properties for better hardware deployment compared to sparse matrices from pruning.
- Multi-Scale Dilation Region Proposal Network (RPN): To improve the recall rate of object proposals and enhance detection accuracy, this RPN incorporates:
- Dilated convolution: Used to expand the receptive field without increasing kernel size or losing feature information, which is beneficial for capturing context.
- Multi-scale feature fusion: Integrates features from different layers (Conv2, Inception3b, Inception4e) to capture a richer representation.
- Global feature weighting module: This module, inspired by Squeeze-and-Excitation networks, adaptively recalibrates channel-wise feature responses, enhancing the network's ability to extract discriminative features.
The feature extractor used in their proposed network is a pruned version of GoogLeNet, specifically up to the inception4e layer.
Datasets
The experiments and evaluations were conducted on the following datasets:
- PASCAL VOC2007: A standard benchmark for object detection.
- PASCAL VOC2012: A larger dataset that builds upon VOC2007.
The authors combined VOC2007 and VOC2012 for training and tested on the PASCAL VOC2007 test set.
Key Insights and Ideas
- Feature Fusion is the Bottleneck: The paper convincingly demonstrates that the feature fusion module in two-stage detectors is the primary contributor to their high computational cost and parameter count. This insight justifies the focus on optimizing this specific component.
- Grouped Convolutions for Efficiency: The use of grouped convolutions and grouped fully connected layers is shown to be an effective strategy for reducing operations and parameters without a significant drop in accuracy, offering a practical approach to model compression.
- Dilated Convolutions for Better Context: Incorporating dilated convolutions in the RPN helps expand the receptive field, allowing the network to capture more contextual information, which is crucial for accurate object proposal generation.
- Global Feature Weighting for Enhanced Representation: The global feature weighting module proves effective in boosting detection accuracy by allowing the network to focus on more informative feature channels.
- Trade-off Achieved: The research successfully demonstrates that it's possible to achieve a good balance between detection accuracy, computational efficiency (operations), and model size, making object detection more deployable.
Key Results
The proposed model, built upon a pruned GoogLeNet, an EFFM, and a multi-scale dilation RPN, achieved significant improvements:
- Accuracy: Achieved an mAP of 72.6% on PASCAL VOC2007 and 72.6% on PASCAL VOC2007 + VOC2012, which is comparable to or slightly higher than Faster R-CNN with VGG16 (72.08%).
- Operations: Reduced operations by approximately half compared to Faster R-CNN based on VGG16 (30.5 G-Ops vs. 51.8 G-Ops).
- Model Size: Reduced the model size to 156 M, which is about one-third of Faster R-CNN based on VGG16 (523 M).
- Speed: Achieved a runtime of 11 frames per second, which is higher than other compared two-stage detectors.
- Parameter Utilization: Demonstrated significantly higher parameter utilization (0.465) compared to Faster R-CNN based on VGG16 (0.138).
- Power Consumption: Showed lower power consumption (137 W) compared to Faster R-CNN based on VGG16 (165 W).
Limitations and Future Work
- Hardware Optimization: The authors acknowledge that the runtime speedup is not fully realized due to Caffe's limited support for certain neural computing units (like grouped convolution and grouped fully connected layers). They believe further optimization is possible with specific hardware.
- Further Compression: There is potential for achieving even higher compression ratios.
- Future Directions: The authors plan to explore model compression techniques like channel pruning and weight pruning in conjunction with their proposed methods. They also aim to deploy their algorithms on specific hardware platforms like Nvidia Jetson TK1, FPGA, and ARM processors to enable wider application on mobile and embedded devices without sacrificing accuracy.
Summary updated