Receptive Field Block Net for Accurate and Fast Object Detection

Songtao Liu, Di Huang, Yunhong Wang

Abstract

The Receptive Field Block Net (RFB Net) addresses the fundamental trade-off between speed and accuracy in real-time object detection. By incorporating biological vision principles, specifically the relationship between receptive field size and eccentricity in human Population Receptive Fields (pRFs), the authors introduce the Receptive Field Block (RFB). This module enhances lightweight feature maps by combining multi-branch convolutions with varying dilation rates. Integrated into a Single Shot Detector (SSD) framework, RFB Net achieves state-of-the-art performance, reaching 80.5% mAP on Pascal VOC 2007 at 83 FPS, effectively bridging the gap between fast one-stage detectors and high-precision two-stage models.

Introduction

Object detection research has traditionally been divided between high-accuracy two-stage detectors (e.g., Faster R-CNN) and high-speed one-stage detectors (e.g., SSD, YOLO). While two-stage models leverage deep backbones and complex proposal phases, one-stage models often struggle with precision due to the limited discriminative power of their lightweight feature extractors.

A significant limitation of current convolutional neural networks (CNNs) is their reliance on uniform sampling grids. In contrast, the human visual system utilizes Population Receptive Fields (pRFs) where the receptive field size increases as a function of eccentricity (distance from the center of vision). This biological structure allows for a more efficient and robust spatial representation.

Methods

The core contribution of this work is the Receptive Field Block (RFB), a multi-branch convolutional module designed to mimic the eccentricity-dependent sampling of the human visual cortex.

The RFB Module

The RFB module employs a two-step approach to simulate biological receptive fields:

  1. Multi-branch Convolution: Similar to Inception modules, the RFB uses parallel branches with different kernel sizes ($1\times1, 3\times3, 5\times5$) to capture information across multiple spatial scales.
  2. Dilated Convolution: To incorporate the concept of eccentricity, the authors apply dilated convolutions to the outputs of the multi-scale kernels. Crucially, higher dilation rates are paired with larger kernels.

This combination results in a "daisy-shaped" sampling pattern, where the central region is sampled more densely than the periphery. This structure produces a more concentrated Effective Receptive Field (ERF), helping the model focus on critical object features.

RFB module architecture and its mapping to the spatial array of pRFs in hV4.

RFB Net Architecture

The authors construct RFB Net by replacing the top convolutional layers of a standard SSD architecture with RFB modules. For earlier, higher-resolution feature maps, a smaller version called RFB-s is utilized to mimic the smaller receptive fields found in the shallow layers of the visual cortex.

Results

The RFB Net was evaluated on the Pascal VOC and MS COCO datasets, demonstrating a superior balance of speed and accuracy.

  • Pascal VOC 2007: RFB Net300 achieved 80.5% mAP at 83 FPS on a Titan X GPU. This significantly outperforms the baseline SSD300 (77.2% mAP) and matches the accuracy of much slower, deeper models like R-FCN.
  • MS COCO: On the challenging COCO test-dev set, RFB Net300 reached 30.3% mAP, comparable to ResNet-101-based R-FCN while maintaining real-time speeds.
  • Generalization: The RFB module's versatility was demonstrated by integrating it with a MobileNet backbone. This addition increased COCO accuracy from 19.3% to 20.7% mAP with negligible impact on computational overhead.

Conclusion

RFB Net demonstrates that bio-inspired architectural choices can significantly enhance the efficiency and accuracy of object detection models. By linking kernel size and dilation rate to simulate the size-eccentricity relationship of human vision, the RFB module provides a powerful mechanism for generating discriminative features in lightweight networks. Future research may explore the application of RFB modules to other computer vision tasks, such as semantic segmentation, or their integration into automated neural architecture search (NAS) frameworks.

Reference: Liu, S., Huang, D., & Wang, Y. (2018). Receptive Field Block Net for Accurate and Fast Object Detection. arXiv:1711.07767v3.

Summary updated

BibTeX
@inproceedings{liu2018receptive,
  author = {Songtao Liu and Di Huang and Yunhong Wang},
  title = {{Receptive Field Block Net for Accurate and Fast Object Detection}},
  booktitle = {European Conference on Computer Vision},
  year = {2018},
  doi = {10.1007/978-3-030-01252-6\_24},
}
Figure