Object Detection Based on Region Decomposition and Assembly

Seung-Hwan BaeAAAI 2019 · DOI

TL;DR

This paper introduces the Region Decomposition and Assembly Detector (R-DAD), a novel object detection framework that improves accuracy by decomposing objects into smaller parts and learning their semantic relationships. It also enhances region proposal generation through a multi-scale approach, leading to state-of-the-art results on challenging benchmarks without relying on complex add-ons like feature pyramids.

What Problem Does This Paper Address?

Object detection, the task of identifying and localizing objects within an image, has seen significant progress with deep learning. However, existing Convolutional Neural Network (CNN)-based detectors often struggle with accuracy degradation due to two main issues:

  1. Inaccurate Region Proposals: Methods that generate potential object locations (region proposals) can be imprecise, especially for small objects or those with significant scale variations. This mismatch between proposed regions and actual object boundaries hinders accurate feature extraction.
  2. Low Feature Discriminability due to Occlusions: When objects are partially occluded, the discriminative power of CNN features extracted from the visible parts can be reduced. Relying solely on global appearance features for the entire object becomes insufficient.

The paper aims to address these limitations by proposing a more robust and accurate object detection system.

Methods

The proposed R-DAD architecture consists of two main novel components:

  1. Multi-Scale Region Proposal (MRP) Network:
    • Instead of relying on a single set of region proposals, the MRP network generates proposals at various scales.
  2. It starts with standard region proposals from a Region Proposal Network (RPN) and then rescales them using different scaling factors (e.g., s = [0.5, 0.7, 1.0, 1.2, 1.5]).
  3. This multi-scale approach helps capture both fine-grained local details (with smaller scales) and broader contextual information (with larger scales), improving robustness to object size variations and occlusions.
  4. A region sampling layer is used to select appropriate proposals for training and inference, managing the data imbalance between foreground and background samples.
  5. Region Decomposition and Assembly (RDA) Network:
    • This component addresses the issue of reduced feature discriminability, particularly for occluded objects.
  6. An object region is decomposed into several smaller "part" regions (e.g., left, right, bottom, upper).
  7. CNN features are extracted from both the whole object region and these decomposed part regions.
  8. A novel Region Assembly Block (RAB) is introduced. RABs use 3x3 convolution filters and ReLU activation to learn semantic relationships between part features. They aggregate maximum responses from part features stage-by-stage, effectively learning how to combine features from different parts adaptively.
  9. This hierarchical assembly allows the model to learn strong semantic dependencies between parts and combine them with global appearance features for more accurate classification and localization.

The R-DAD architecture integrates these components into existing frameworks like Faster R-CNN, demonstrating its flexibility.

Datasets

The paper evaluates its proposed R-DAD on the following standard object detection benchmarks:

  • PASCAL VOC 2007 (VOC07)
  • PASCAL VOC 2012 (VOC12)
  • MS COCO 2018 (MSCOCO18)

The authors use VOC07++12 for training and VOC2012test for evaluation. For MSCOCO, they use the union of train and validation images for training and the test-dev set for evaluation.

Key Insights and Ideas

  • Decomposition for Robustness: Breaking down an object into parts and learning their relationships is crucial for handling occlusions. The model can learn to rely more on visible parts when others are hidden.
  • Multi-Scale Proposals are Key: Generating region proposals at various scales significantly improves the ability to detect objects of different sizes and capture both local details and global context. This is achieved without the computational overhead of image pyramids.
  • Adaptive Feature Combination: The Region Assembly Block (RAB) provides a mechanism to adaptively combine features from different object parts, allowing the model to learn which parts are most informative under varying conditions.
  • Simplicity and Effectiveness: The R-DAD achieves state-of-the-art results without relying on common performance boosters like feature pyramids, multi-scale testing, data augmentation, or model ensembles, highlighting the inherent effectiveness of its core design.

Key Results

The R-DAD demonstrates significant performance improvements across various benchmarks:

  • PASCAL VOC 2007/2012: Achieved state-of-the-art mean Average Precision (mAP) scores, with improvements of up to 11.6% over Faster R-CNN and 8.1% over MR-CNN. For example, using a ResNet152 model, they achieved an mAP of 81.2% on VOC07.
  • MS COCO 2018: Achieved competitive results, with an mAP of 43.1% using ResNet101. The paper highlights that their R-DAD achieved the best scores without common "bells and whistles."
  • Speed: The detection speed of R-DAD is comparable to Faster R-CNN, indicating that the added complexity of MRP and RDA does not significantly increase inference time. For instance, on ZF, R-DAD takes 0.048 sec/frame compared to FRCN's 0.041 sec/frame.
  • Ablation Studies:
    • Using multi-scale proposals with five scales improved mAP by 3.05%.
  • Up-sampling feature maps improved mAP by 1%.
  • Combining all proposed methods (MRP and RDA) with a baseline detector enhanced mAP by 6%.

Limitations and Future Work

  • Speed Trade-off: While comparable, R-DAD is slightly slower than Faster R-CNN in some configurations (e.g., 1.5x-2.1x slower when using m=5 in RABs, though this yielded only a minor mAP improvement). The authors note that further optimization might be needed for real-time applications.
  • Future Work: The paper suggests that the R-DAD architecture is flexible and can be applied to other convolutional detectors and region proposal networks, implying potential for broader integration and further research. The authors also mention that the spatial sizes of RoI feature maps and convolution filters can affect speed, suggesting avenues for optimization.

Summary updated

BibTeX
@inproceedings{bae2019object,
  author = {Seung-Hwan Bae},
  title = {{Object Detection Based on Region Decomposition and Assembly}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2019},
  doi = {10.1609/aaai.v33i01.33018094},
}
Figure