StairNet: Top-Down Semantic Aggregation for Accurate One Shot Detection

Sanghyun Woo, Soonmin Hwang, In So Kweon

Abstract

One-stage object detectors like SSD and YOLO offer impressive speed and a small memory footprint, yet they often struggle to detect small objects compared to their two-stage counterparts. This performance gap stems from the pyramidal design of these networks: lower layers, which are responsible for small objects, lack the strong semantic context found in deeper layers. In this paper, we introduce StairNet, a detector that alleviates this problem by employing a feature combining module to propagate high-level semantics in a top-down manner. By unifying multi-scale representations and semantic distribution, StairNet significantly improves upon the SSD framework, outperforming other state-of-the-art one-stage detectors on PASCAL VOC benchmarks.

The Semantic Gap in One-Stage Detectors

In computer vision, object detectors generally fall into two categories: accurate but slow two-stage detectors (e.g., Faster R-CNN) and fast but less precise one-stage detectors (e.g., SSD). The standard Single Shot MultiBox Detector (SSD) uses a pyramid of feature maps to detect objects at various scales.

However, this architecture has a fundamental "weak spot." Deep layers at the top of the pyramid possess a global view and "strong semantics" (context), while shallower layers at the bottom—responsible for small objects—only see local details. These bottom layers lack the contextual "big picture" required to accurately identify small, truncated, or overlapping objects. StairNet was designed specifically to bridge this semantic gap.

Methodology: The StairNet Architecture

StairNet introduces two primary architectural innovations to the standard SSD framework to harmonize features across scales.

Feature Combining Module (FCM)

The heart of StairNet is the Feature Combining Module. Instead of allowing each layer to predict independently, StairNet implements a top-down path. It extracts semantically rich features from a higher layer, upsamples them using deconvolution, and merges them with the features of the layer below via element-wise addition.

The StairNet architecture (bottom left) compared to standard SSD (top left), with a detailed view of the Feature Combining Module (right).

A final 3x3 convolution is applied after the summation to "smooth" the combined features before prediction. This acts as a buffer that harmonizes features from different depths, making training more stable.

Unified Prediction Layer

Unlike the original SSD, which uses distinct classifiers for each scale, StairNet employs a shared (unified) classifier across all levels. Because the FCM makes features across different scales semantically similar, a single shared classifier is more efficient and helps mitigate data imbalance issues, such as having many large training examples but few small ones for a specific category.

Key Insights and Results

Experiments on the PASCAL VOC 2007 and 2012 datasets demonstrate that StairNet effectively addresses the weaknesses of the baseline SSD.

  • Accuracy Boost: On PASCAL VOC 2007, StairNet achieved 78.8% mAP, outperforming the baseline SSD (77.2%) and other competitors like DSSD.
  • Small Object Mastery: The most significant gains were in small object detection, where StairNet showed an 8.9 mAP increase over the original SSD. By "pouring" global context from top layers down to the bottom, the network identifies objects that were previously indistinguishable from noise.
  • Real-Time Performance: Despite the added complexity of deconvolution layers, StairNet maintains a speed of 30 FPS on an NVIDIA Titan X GPU, keeping it well within the threshold for real-time applications.
Qualitative comparison showing StairNet's ability to detect small and occluded objects (yellow boxes) that the baseline SSD (red boxes) missed.

Conclusion

StairNet demonstrates that the weakness of one-stage detectors in handling small objects is not inherent to the one-stage approach itself, but rather a consequence of poor semantic distribution. By using a staircase of feature-combining modules and learned upsampling, StairNet provides a robust, real-time solution for accurate multi-scale object detection.

Summary updated

Abstract

One-stage object detectors such as SSD or YOLO already have shown promising accuracy with small memory footprint and fast speed. However, it is widely recognized that one-stage detectors have difficulty in detecting small objects while they are competitive with two-stage methods on large objects. In this paper, we investigate how to alleviate this problem starting from the SSD framework. Due to their pyramidal design, the lower layer that is responsible for small objects lacks strong semantics(e.g contextual information). We address this problem by introducing a feature combining module that spreads out the strong semantics in a top-down manner. Our final model StairNet detector unifies the multi-scale representations and semantic distribution effectively. Experiments on PASCAL VOC 2007 and PASCAL VOC 2012 datasets demonstrate that StairNet significantly improves the weakness of SSD and outperforms the other state-of-the-art one-stage detectors.
BibTeX
@inproceedings{woo2018stairnet,
  author = {Sanghyun Woo and Soonmin Hwang and In So Kweon},
  title = {{StairNet: Top-Down Semantic Aggregation for Accurate One Shot Detection}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2018},
  url = {https://openreview.net/forum?id=QtQxbSdokF},
}
Figure