Overview
This paper introduces a two-stage, RNN-based framework designed to replace the traditional "Non-Maximum Suppression" (NMS) step in object detection. By treating object proposals as a sequence and using global attention and context gates, the model learns to intelligently select the single best bounding box for an object rather than relying on simple greedy heuristics.
The Problem with NMS
In modern object detection architectures (like Faster R-CNN or FPN), the model generates thousands of candidate boxes, or "proposals," that might contain an object. After classifying these boxes, the system is often left with dozens of overlapping detections for a single object.
The standard solution is Duplicate Removal, typically handled by Non-Maximum Suppression (NMS). NMS is a simple, greedy algorithm: it picks the box with the highest score, deletes everything that overlaps with it significantly, and repeats. However, NMS is "dumb"—it doesn't look at the actual visual features of the boxes or the global context of the image. This paper addresses this often-overlooked area by proposing a learned, context-aware alternative.
A Two-Stage Sequential Framework
The authors propose a two-stage sequential framework that uses Recurrent Neural Networks (RNNs) to "read" through proposals and decide which ones to keep.
- Feature Embedding: For every proposal, the model extracts four types of information:
- Appearance: High-dimensional features from the detection network.
- Geometry: The coordinates and size of the box.
- Score: The classification confidence.
- Rank: The proposal's rank when sorted by score (using positional encoding).
- Two-Stage Cascade:
- Stage I: Quickly suppresses "easy" negatives (obvious duplicates or background).
- Stage II: Performs a finer selection to find the most accurate "true positive" box among the remaining candidates.
- Encoder-Decoder Architecture:
- The Encoder (a bi-directional GRU) scans the proposals in order of their scores to capture global context.
- The Decoder re-scans the proposals, using a Global Attention mechanism to weigh the relationship between different candidates.
- A Context Gate dynamically balances low-level features with high-level global context to make the final decision.
Key Insights
- Sorting Matters: RNNs usually process temporal sequences. To make them work for spatial boxes, the authors sorted proposals by confidence score. This allows the network to "see" the most likely candidates first, mirroring how a human might scan a list.
- Context is King: Traditional NMS only looks at two boxes at a time (IoU overlap). This model looks at the entire set of proposals. If the model knows there are five other "dog" proposals nearby, it can better judge which one is the most centered.
- The Power of Rank: Simply knowing a box is the "3rd best" vs. the "100th best" is a powerful signal for the network, captured using positional encoding similar to Transformers.
- Efficiency: Despite using an RNN, the overhead is tiny—only about 1% of the total parameters of a standard FPN model, adding just ~0.019s to the inference time.
Results on MS COCO
The model was evaluated on the MS COCO dataset across three major detection baselines: FPN, Mask R-CNN, and PANet. The proposed method consistently outperformed NMS and other variants like Soft-NMS and Box Voting:
- mAP Gains: On the FPN baseline, the method improved Mean Average Precision (mAP) from 37.1 to 38.6.
- Localization Accuracy: The improvements were even more pronounced at higher overlap thresholds ($mAP_{75}$), suggesting the model is much better at picking the most precise box than NMS.
- Proposal Reduction: Qualitatively, the method produces far fewer redundant boxes without losing accuracy.
Limitations and Future Work
While the computational overhead is small, RNNs are inherently sequential, which can be harder to parallelize than the rest of the detection pipeline. Furthermore, the current model processes each object class independently. Future work could model relationships between different classes (e.g., a "keyboard" proposal should be near a "monitor" proposal). Ultimately, the authors aim to integrate this directly into the detection network's training loss for a truly end-to-end "one-shot" detector.
Summary updated