TL;DR
This paper introduces SSD-EMB, an enhanced version of the Single Shot MultiBox Detector (SSD) that improves small object detection accuracy without sacrificing speed. It achieves this by introducing an "Enhanced Map Block" (EMB) that combines an attention mechanism to focus on relevant regions with a feature map concatenation stream to enrich semantic information.
What Problem Does This Paper Address?
Object detection, a fundamental task in computer vision with applications ranging from autonomous driving to medical imaging, has seen significant progress thanks to deep learning. However, accurately detecting small objects within an image remains a persistent challenge. Existing methods often struggle to capture the fine details of small objects, leading to lower accuracy. While some approaches improve small object detection, they frequently do so at the expense of computational speed, making them unsuitable for real-time applications. This paper aims to bridge this gap by proposing a method that enhances small object detection accuracy while maintaining high detection speeds.
Methods
The core innovation of this paper is the Enhanced Map Block (EMB), which is integrated into the SSD architecture. The EMB comprises two key components:
- Attention Stream: This stream uses channel-wise average pooling and a sigmoid activation function to generate an "importance map." This map highlights regions of interest within the feature map, effectively guiding the model to focus on potential object locations and reducing the influence of background noise. Crucially, this process requires no additional learning, preserving detection speed.
- Feature Map Concatenation Stream: This stream addresses the issue of shallow feature maps having less semantic information. It splits the input feature map in half, processes one half through three convolutional layers to extract richer semantic features, and then concatenates it with the other half. This approach leverages a reduced number of parameters compared to standard convolutions, mitigating speed degradation.
These two streams are combined, and the resulting enhanced feature map is added to the original input feature map. This process enriches the feature representation, particularly for small objects, by combining focused attention with enhanced semantic context. The SSD-EMB architecture applies these EMBs to specific convolutional layers of the SSD, typically the shallower ones where small objects are more likely to be missed.
Datasets
The proposed SSD-EMB model was evaluated on three widely used benchmark datasets:
- PASCAL VOC 2007: A standard dataset for object detection, containing 20 object categories.
- PASCAL VOC 2012: An extension of VOC 2007, offering a larger number of images and categories.
- MS COCO (Microsoft Common Objects in Context): A more challenging dataset with 80 object categories, known for containing a higher proportion of small objects.
Key Insights and Ideas
- The "Attention + Concatenation" Synergy: The paper demonstrates that combining an attention mechanism (to focus on where to look) with a feature map concatenation stream (to enrich what is seen) is more effective than using either stream in isolation for improving object detection.
- Shallow Layers are Key for Small Objects: The research highlights that applying enhancements to the feature maps from shallower convolutional layers is particularly effective for improving small object detection. These layers retain higher spatial resolution, which is crucial for capturing the details of small objects.
- No Learning for Attention: The fact that the attention stream operates without additional learned parameters is a significant design choice. This allows the model to gain the benefits of focused attention without incurring the computational overhead typically associated with attention mechanisms.
- Balancing Accuracy and Speed: The core contribution is achieving a better trade-off between detection accuracy (especially for small objects) and inference speed, a common challenge in object detection research.
Key Results
The SSD-EMB model demonstrated strong performance across the evaluated datasets:
- PASCAL VOC 2007: Achieved a mean average precision (mAP) of 78.4% for the SSD300 variant and 80.4% (as stated in the abstract, though Table 1 shows 80.2% for SSD512-EMB) for the SSD512 variant, showing improvements over the original SSD300 (77.2%) and SSD512 (79.5%). Notably, it maintained a detection speed of 30 frames per second (FPS) on an RTX 2080Ti GPU.
- PASCAL VOC 2012: The SSD-EMB achieved an mAP of 79.9% for the SSD512 variant, outperforming the original SSD512.
- MS COCO: Achieved an mAP of 26.6% (for the [0.5:0.95] IoU metric), showing a modest improvement over the original SSD300 (25.1%), though it was outpaced by models using stronger backbones like ResNet-101.
The paper also includes ablation studies showing that applying EMBs to multiple shallow layers (Conv4_3, Conv7, Conv8_2) yielded the best results, and that the combined attention and concatenation streams were superior to using either alone.
Limitations and Future Work
- Backbone Dependency: On the challenging MS COCO dataset, the performance improvement of SSD-EMB was limited by the choice of backbone network (VGG). The authors note that using stronger backbones like ResNet-101 or DenseNet-S-32-1 could lead to greater accuracy gains, though this might impact speed.
- Occlusion and Similar Objects: The paper acknowledges that detecting occluded objects or objects with very similar appearances can still be challenging, leading to occasional incorrect predictions.
- Future Work: The authors plan to explore applying EMBs to other state-of-the-art networks, develop new attention mechanisms, and investigate low-rank approximation techniques to further improve classification accuracy. They also express interest in exploring weakly supervised object detection methods, which rely on less precise annotations.
Summary updated