Foreground Feature Enhancement for Object Detection

Shenwang Jiang, Tingfa Xu, Jianan Li, Ziyi Shen, Jie GuoIEEE Access 2019 · OpenReview

Abstract

Deep convolutional neural networks have shown great success in object detection. Most object detection methods focus on improving network architecture and introducing additional objective functions to improve the discrimination of object detectors, while the informative annotations of the training data obtained from enormous human effort are mainly used in the last stage of the network for producing supervisions, thus being under-explored. In this paper, we propose to take further advantage of bounding box annotations to highlight the feature map of foreground objects by erasing background noise with a novel Mask loss, in which process $L_2$ norm is further incorporated to avoid degenerated features. The extensive experiments on PASCAL VOC 2007, VOC 2012, and COCO 2017 will demonstrate the proposed method can greatly improve detection performance compared with baseline models, thus achieving competitive results.

Introduction

In modern object detection, human-labeled bounding boxes represent a significant investment of effort. However, these annotations are typically utilized only at the final stage of a network to calculate loss. This leaves a wealth of information under-utilized during the feature extraction process. A persistent challenge in detection is "background noise"—clutter such as shadows and textures that can obscure or confuse the features of the target object. While standard networks attempt to ignore this noise implicitly, this research proposes an explicit mechanism to "teach" the network to focus on foreground features by leveraging existing annotations more effectively.

Methodology: Siamese Foreground Enhancement

The proposed framework utilizes a Siamese network architecture based on a Faster R-CNN backbone (ResNet101). It consists of three distinct sub-networks:

  1. ConvNet_teacher: This branch processes a "foreground-only" version of the input image, where background pixels (those outside the ground-truth bounding boxes) are replaced by the image's mean pixel value. This allows the teacher to learn idealized foreground features.
  2. ConvNet_student: This branch receives the original, full image. Its objective is to extract features that mimic the teacher's "clean" foreground representations.
  3. ConvNet_normal: A standard detection branch that processes the full image to ensure the network maintains a general representation of the scene.
The proposed Siamese network architecture featuring Teacher, Student, and Normal ConvNet branches, integrated with Mask and Detection losses.

The core innovation is the Mask loss, defined as a normalized $L_2$ distance between the feature maps of the Teacher and Student branches. By minimizing this loss, the Student branch learns to internally suppress background noise. To prevent the feature maps from becoming overly blurred—a common issue with $L_2$ optimization—the authors apply this constraint across multiple adjacent layers (specifically conv3 and conv4). The resulting enhanced foreground features are then combined with the normal branch's features via Feature Fusion (summation) before the final detection stage.

Experimental Results

The effectiveness of the foreground enhancement method was validated across three major benchmarks:

  • PASCAL VOC 2007: The model achieved a mean Average Precision (mAP) of 79.9%, a significant +3.5% improvement over the 76.4% baseline.
  • PASCAL VOC 2012: Performance increased from 73.2% to 77.2% mAP.
  • MS COCO 2017: The method improved mAP from 32.4% to 34.2%. Notably, gains were observed across all object scales, including a 2.2% increase for large objects.

Qualitative analysis via saliency maps confirmed that the proposed model produces tighter, more concentrated activations on the objects themselves compared to the sparse, noise-prone activations of the baseline model.

Conclusion

By treating bounding box annotations as guides for intermediate feature distillation rather than just final targets, this method significantly boosts detection accuracy. While the $L_2$ norm introduces some blurring risks, the multi-layer constraint effectively mitigates this. Future work may explore alternative loss functions to further refine foreground isolation and extend this logic to other architectures.

Summary updated

Abstract

Deep convolutional neural networks have shown great success in object detection. Most object detection methods focus on improving network architecture and introducing additional objective functions to improve the discrimination of object detectors, while the informative annotations of the training data obtained from enormous human effort are mainly used in the last stage of the network for producing supervisions, thus being under-explored. In this paper, we propose to take further advantage of bounding box annotations to highlight the feature map of foreground objects by erasing background noise with a novel Mask loss, in which process L 2 norm is further incorporated to avoid degenerated features. The extensive experiments on PASCAL VOC 2007, VOC 2012, and COCO 2017 will demonstrate the proposed method can greatly improve detection performance compared with baseline models, thus achieving competitive results.
BibTeX
@inproceedings{jiang2019foreground,
  author = {Shenwang Jiang and Tingfa Xu and Jianan Li and Ziyi Shen and Jie Guo},
  title = {{Foreground Feature Enhancement for Object Detection}},
  booktitle = {IEEE Access},
  year = {2019},
  url = {https://openreview.net/forum?id=meZOWgolrl},
}
Figure