Introduction
Small object detection remains a significant challenge in computer vision. While state-of-the-art detectors like Faster R-CNN perform exceptionally well on standard-sized objects, their accuracy drops sharply when objects occupy only a few pixels. This performance gap stems from two primary issues: information loss during downsampling and feature distortion caused by misalignment during pooling operations.
The Challenge of Precise Supervision
A common strategy to combat information loss is Super-Resolution (SR), which aims to recover high-resolution (HR) details from low-resolution (LR) inputs. However, applying SR to object detection features is non-trivial. Previous methods often relied on indirect supervision from detection losses or used naive HR features as targets.
The authors identify a critical "Relative Receptive Field (RRF) mismatch" in these approaches. A small object in an LR image inherently captures more surrounding context relative to its size than a large object in an HR image. Forcing an LR feature to mimic an HR feature without accounting for this difference leads to imprecise supervision and suboptimal performance.
Proposed Methodology: Precise Feature Super-Resolution
To address these challenges, the authors propose a GAN-based framework integrated into the Faster R-CNN pipeline. The architecture is built around several key components:
SR Target Extractor
The core innovation is the SR Target Extractor, which serves as the "teacher" network. By utilizing atrous (dilated) convolutions, it ensures that the high-resolution target features encompass the same relative context as the low-resolution input features. This alignment provides a precise supervision signal that resolves the RRF mismatch.
SR Feature Generator and Adversarial Training
The SR Feature Generator is a GAN-based module that upscales LR features. It doesn't just upscale the final layer; it incorporates "sub-layer" features from earlier in the backbone to provide fine-grained structural blueprints. A discriminator is employed to distinguish between generated SR features and real features from the Target Extractor, driving the generator toward high-fidelity outputs.
Dual Predictors
The model employs a split workload strategy. Large objects are processed by a standard "Large Predictor," while small objects are enhanced by the SR Generator before being handled by a specialized "Small Predictor." This ensures that the model can optimize for different scales without compromise.
Experimental Results
The framework was evaluated on three major benchmarks: Tsinghua-Tencent 100K (TT100K), PASCAL VOC 2007, and MS COCO.
- TT100K: On this traffic sign dataset, where objects are extremely small, the model achieved a massive improvement. Using a ResNet-101 backbone, the F1 score for small objects jumped from 75.2 to 84.3.
- MS COCO: The approach demonstrated consistent gains across all object sizes, proving that improving small-object features does not negatively impact the detection of larger objects.
- Ablation Studies: The results confirmed that without the atrous convolutions in the Target Extractor (i.e., using naive targets), the SR process could actually degrade performance, highlighting the importance of precise supervision.
Conclusion
This work demonstrates that effective feature super-resolution for small object detection requires more than just upscaling; it requires precise alignment of receptive fields. By introducing the SR Target Extractor with atrous convolutions, the authors provide a robust framework for supervising the generation of high-fidelity features, leading to state-of-the-art results on challenging datasets.
Summary updated