Dual-Resolution Dual-Path Convolutional Neural Networks for Fast Object Detection

Jing Pan, Hanqing Sun, Zhanjie Song, Jungong Han2019 · DOI

TL;DR

This paper introduces DualNets, a novel dual-resolution, dual-path convolutional neural network architecture designed to improve object detection accuracy in computationally constrained environments. By processing images at two different resolutions and progressively fusing features, DualNets achieve higher accuracy than standard single-resolution models without a significant increase in computational cost.

What Problem Does This Paper Address?

Real-time object detection is crucial for applications like robotics and autonomous systems. While Convolutional Neural Networks (CNNs) have revolutionized object detection, their high computational demands often make them unsuitable for embedded devices. A common trick to reduce computational cost is downsampling input images, but this significantly degrades detection accuracy. This paper addresses the trade-off between speed and accuracy, aiming to boost the accuracy of fast, computationally-efficient object detectors without incurring the "squared" computational cost associated with simply increasing input resolution.

Methods

The core innovation is the DualNets architecture, which employs a dual-resolution, dual-path approach.

  • Dual Inputs: DualNets take two inputs: a lower-resolution image for the "backbone" path and a higher-resolution image for the "auxiliary" path.
  • Dual Paths:
    • Backbone Path: This path uses a standard, efficient CNN architecture (like MobileNetV2) processing the lower-resolution input.
  • Auxiliary Path: This path is designed to be lightweight, with fewer layers and rapid downsampling. It processes the higher-resolution input to extract complementary visual features.
  • Progressive Residual Fusion: This is the key fusion strategy. Instead of fusing features only at the end, DualNets progressively fuse features from the auxiliary path into the backbone path. This is guided by a novel residual-learning formula: $H(x) = (F(x_s) + x_s) + R(x_l)$, where $x_s$ is the small input, $x_l$ is the large input, $F$ is the backbone's learned mapping, and $R$ is the residual mapping learned by the auxiliary path. This strategy aims to diminish capacity gaps between the asymmetric paths and make learning easier for the auxiliary path.
  • Progressive Early Fusion: The paper explores different fusion timings, concluding that "progressive early fusion" (fusing features as early as possible in the backbone path when dimensions match) yields the best results.

Datasets

  • Pascal VOC: Used for ablation studies and evaluation. Specifically, Pascal VOC 2007 and 2012 (trainval0712) for training and Pascal VOC 2007 (test2007) for validation.
  • MS COCO: Used for evaluating the proposed DualNet-300 and DualNet-512. Trained on trainval35k and evaluated using the test-dev2018 dataset.

Key Insights and Ideas

  • Asymmetric Dual Paths are Beneficial: The paper argues that forcing dual paths to share weights (as in some prior work) is detrimental when the paths are inherently asymmetric in resolution and structure.
  • Progressive Fusion is Crucial: Simply fusing features at the end of the network is not as effective as progressively integrating auxiliary features into the backbone. This allows the backbone to learn from enriched representations throughout its layers.
  • Residual Learning for Fusion: The novel residual learning formula is key to effectively integrating features from the auxiliary path. It treats the auxiliary path's contribution as a residual to be learned, making it easier to fit and diminishing the performance gap caused by path asymmetry.
  • Controllable Computation: The auxiliary path's lightweight design and rapid downsampling allow for flexible control over computational cost, making the approach adaptable to various embedded systems.

Key Results

  • Pascal VOC: DualNet-300 achieved an mAP of 67.01% (at mIoU threshold 0.5) with a 100k iteration training, outperforming MobileNetV2-SSD-300 (66.48%) while maintaining competitive inference times.
  • MS COCO:
    • DualNet-300 achieved higher mAP scores across various metrics compared to MobileNetV2-SSD-300, particularly for large-scale objects.
  • DualNet-512 also showed improvements over MobileNetV2-SSD-512, demonstrating the scalability of the approach.
  • Ablation Studies:
    • Weight sharing between asymmetric paths was found to be harmful.
  • Random initialization of the auxiliary path outperformed weight sharing and pre-trained initialization, supporting the idea that the residual learning strategy can effectively bridge the gap between asymmetric paths.
  • Progressive early fusion was shown to be superior to late fusion and simple detection-time fusion.

Limitations and Future Work

  • Limitations:
    • The paper notes that while DualNets improve accuracy, they still struggle with extremely challenging cases like ambiguous, occluded, or truncated objects, which remains a general challenge for object detection in embedded systems.
  • The training process for DualNets, especially for larger resolutions like 512x512, might still be computationally intensive, and the paper mentions using a single GPU and faster learning rate decay due to resource limitations, which could lead to performance drops compared to models trained with more resources.
  • Future Work:
    • The authors plan to apply their object detector to facilitate real-world applications such as image retrieval, image tracking, and image classification.
  • The framework is presented as flexible and could be applied to other small detection models sensitive to computation payloads.

Summary updated

BibTeX
@misc{pan2019dual,
  author = {Jing Pan and Hanqing Sun and Zhanjie Song and Jungong Han},
  title = {{Dual-Resolution Dual-Path Convolutional Neural Networks for Fast Object Detection}},
  year = {2019},
  doi = {10.3390/s19143111},
}

Loading related content...

Figure