Multi-stage Object Detection with Group Recursive Learning

Jianan Li, Xiaodan Liang, Jianshu Li, Tingfa Xu, Jiashi Feng, Shuicheng Yan

Abstract

This paper presents a multi-stage object detection framework designed to enhance localization accuracy by integrating weakly-supervised semantic segmentation and a novel Group Recursive Learning (GRL) strategy. Unlike traditional detectors that process object proposals in isolation, the proposed model allows neighboring proposals to refine their locations collectively. By leveraging spatial correlations and semantic context, the framework achieves state-of-the-art results on PASCAL VOC benchmarks, demonstrating the power of collaborative refinement in computer vision tasks.

Introduction

Most contemporary object detection pipelines, such as the R-CNN family, treat object proposals as independent entities. This approach overlooks two critical sources of information: spatial correlation and semantic context. Multiple proposals often overlap the same object, and their collective geometry can provide a stronger hint about true boundaries than any single box. Furthermore, distinguishing foreground pixels from the background (segmentation) is inherently useful for precise localization. This research addresses these gaps by moving toward a collaborative, multi-stage refinement process.

Methodology

The proposed architecture consists of three cascaded networks designed to work in a unified pipeline:

  1. Weakly-Supervised Segmentation Network: This stage identifies foreground pixels using only bounding box labels, avoiding the need for expensive pixel-level masks. These segmentation features (SF) provide essential local cues about object boundaries.
  2. Proposal Generation Network: Building on the standard Region Proposal Network (RPN), this stage incorporates the segmentation features from the first network to generate higher-quality initial candidates.
  3. Recursive Detection Network with GRL: This is the core innovation of the framework. Instead of a single pass of bounding box regression, the network performs iterative refinement.
The unified architecture showing the flow from the Deep ConvNet through segmentation and proposal generation to the recursive detection network.

Group Recursive Learning (GRL)

Inspired by the Expectation-Maximization (EM) algorithm, GRL groups neighboring proposals that likely correspond to the same object. In each iteration, a proposal's location is refined using a weighted average of its own predicted coordinates and those of its neighbors in the group, weighted by their classification scores. This "Group Confidence Pooling" allows the most accurate boxes to guide the weaker ones toward the ground truth.

A comparison of single recursive refinement versus the proposed group recursive refinement, illustrating how GRL better aligns proposals with the ground truth.

Key Insights

  • The "Committee" Effect: GRL treats overlapping proposals as a committee. The consensus—a weighted average of multiple predictions—is often significantly closer to the actual object than any individual box's prediction.
  • Segmentation as a Guide: Incorporating segmentation features helps the network distinguish object boundaries from the background, which is particularly effective for small or occluded objects.
  • Iterative Refinement: The authors found that unrolling the detection network into two iterations ($T=2$) provided the optimal balance between accuracy and efficiency, allowing the model to progressively "tighten" its localization.

Experimental Results

The model was evaluated on the PASCAL VOC 2007 and 2012 benchmarks, outperforming established baselines:

  • PASCAL VOC 2007: Achieved 78.6% mAP, surpassing Faster R-CNN (73.2%) and MR-CNN (78.2%).
  • PASCAL VOC 2012: Achieved 74.9% mAP, a significant lead over Faster R-CNN's 70.4%.
  • Ablation Studies: Results confirmed that segmentation features added ~0.4% mAP, while GRL contributed an additional 1.2% to 2.2% depending on the specific configuration.

Conclusion

The integration of Group Recursive Learning and semantic segmentation represents a significant step forward in multi-stage object detection. While the iterative approach introduces some computational overhead, the gains in precision are substantial. Future work may extend this recursive logic to instance segmentation or video object detection, where temporal groups of boxes could be refined across frames.

Summary updated

Abstract

Most existing detection pipelines treat object proposals independently and predict bounding box locations and classification scores over them separately. However, the important semantic and spatial layout correlations among proposals are often ignored, which are actually useful for more accurate object detection. In this paper, we propose a new EM-like group recursive learning approach to iteratively refine object proposals by incorporating such context of surrounding proposals and provide an optimal spatial configuration of object detections. In addition, we propose to incorporate the weakly supervised object segmentation cues and region-based object detection into a multistage architecture in order to fully exploit the learned segmentation features for better object detection in an end-toend way. The proposed architecture consists of three cascaded networks that, respectively, learn to perform weakly supervised object segmentation, object proposal generation, and recursive detection refinement. Combining the group recursive learning and the multistage architecture provides competitive mAPs of 78.7% and 74.9% on the PASCAL VOC2007 and VOC2012 datasets, respectively, which outperform many well-established baselines significantly.
BibTeX
@article{li2018multi,
  author = {Jianan Li and Xiaodan Liang and Jianshu Li and Tingfa Xu and Jiashi Feng and Shuicheng Yan},
  title = {{Multi-stage Object Detection with Group Recursive Learning}},
  journal = {IEEE Trans. Multimedia},
  year = {2018},
  doi = {10.1109/tmm.2017.2772796},
}
Figure