Abstract: Region-based Convolutional Neural Networks (R-CNNs) have achieved great success in the field of object detection. The existing R-CNNs usually divide a Region-of-Interest (ROI) into grids, and then localize objects by utilizing the spatial information reflected by the relative position of each grid in the ROI. In this paper, we propose a novel feature-encoding approach, where spatial information is represented through the spatial distributions of visual patterns. In particular, we design a Mask Weight Network (MWN) to learn a set of masks and then apply channel-wise masking operations to ROI feature map, followed by a global pooling and a cheap fully-connected layer. We integrate the newly designed feature encoder into the Faster R-CNN architecture. The resulting new Faster R-CNNs can preserve the object-detection accuracy of the standard Faster R-CNNs by using substantially fewer parameters. Compared to R-FCNs using state-of-art PS ROI pooling and deformable PS ROI pooling, the new Faster R-CNNs can produce higher object-detection accuracy with good run-time efficiency. We also show that a specifically designed and learned MWN can capture global contextual information and further improve the object-detection accuracy. Validation experiments are conducted on both PASCAL VOC and MS COCO datasets.
Introduction
In the field of object detection, Region-based Convolutional Neural Networks (R-CNNs) face a fundamental architectural challenge: balancing translation invariance with translation variance. While deep classification backbones like ResNet are designed to be translation-invariant (recognizing a category regardless of its position), detection requires translation variance to accurately localize object parts.
Standard architectures like Faster R-CNN address this by employing massive Fully Connected (FC) layers after ROI pooling, which are computationally expensive. Alternatives like R-FCN utilize "Position-Sensitive" grids, but these grids are often rigid and arbitrary. This paper proposes that it is more natural to represent object parts through the spatial distribution of visual patterns rather than forced grid cells.
Proposed Method: Mask-based Feature Encoding
The core innovation is the Mask Weight Network (MWN), which replaces rigid spatial grids with learned, channel-wise masks. The proposed architecture, termed M-FRCN (MWN-based Faster R-CNN), follows a streamlined pipeline:
- ROI Pooling: An ROI is pooled from the full-image feature map to a fixed size (e.g., $7 \times 7$).
- Mask Learning: The MWN processes a "raw mask" through a convolution layer to produce a set of learned masks ($M$), with one mask corresponding to each feature channel.
- Channel-wise Masking: Each channel in the ROI feature map—representing specific visual patterns like a "head" or "wheel"—is element-wise multiplied by its learned mask.
- Global Max Pooling (GMP): The masked features are collapsed via GMP, followed by a small, "cheap" FC layer for final classification and bounding-box regression.
The authors introduce two variants: M-FRCN-l (Local), which learns masks for features strictly inside the ROI, and M-FRCN-g (Global), which projects the ROI's position relative to the entire image to capture surrounding context.
Experimental Results
The approach was validated on the PASCAL VOC and MS COCO datasets, demonstrating significant improvements in both efficiency and accuracy.
Parameter Efficiency
By shifting spatial encoding into masks and utilizing Global Max Pooling, the model drastically reduces parameter counts. For a VGG-16 backbone, the parameters in the detection head dropped from 102 million in a standard Faster R-CNN to just 131,000, without sacrificing performance.
Detection Accuracy
On the MS COCO benchmark with a ResNet-101 backbone, the combined M-FRCN-lg model achieved 51.9% [email protected]. This outperforms the standard R-FCN (48.9%) and the more complex Deformable R-FCN (50.0%). The inclusion of global context (M-FRCN-g) proved particularly effective for detecting small or obscured objects by leveraging spatial relationships within the full image.
Runtime Performance
M-FRCN-lg maintains high efficiency, clocking in at 0.17 sec/img on ResNet-101, which is faster than Deformable R-FCN (0.23 sec/img) due to the avoidance of expensive bilinear interpolation operations.
Conclusion
This paper demonstrates that mask-based feature encoding is a superior alternative to traditional grid-based ROI pooling. By leveraging the fact that CNN channels already act as part detectors, the Mask Weight Network allows for a more flexible and parameter-efficient representation of spatial information. Future work may explore dynamic masks that adapt to image content in real-time or the integration of this encoding scheme into one-stage detectors like YOLO or SSD.
Summary updated