ViS-HuD: Using Visual Saliency to Improve Human Detection With Convolutional Neural Networks

Vandit Gajjar, Yash Khandhediya, Ayesha Gurnani, Viraj Mavani, Mehul S. RavalWorkshops @ CVPR 2018 · DOI

Abstract

Human detection in cluttered and occluded real-world environments remains a significant challenge for standard Convolutional Neural Networks (CNNs). This paper introduces ViS-HuD (Visual Saliency for Human Detection), a novel approach that utilizes visual saliency maps as "soft masks" to prioritize relevant image regions. By multiplying input images with their corresponding saliency maps before processing them through a DetectNet architecture, the model effectively suppresses background noise. Experimental results on the Penn-Fudan and TUD-Brussels datasets demonstrate that ViS-HuD achieves state-of-the-art performance, particularly in complex "in-the-wild" scenarios.

Introduction

Human detection is a fundamental task in computer vision, yet its performance often degrades in "in-the-wild" scenarios characterized by cluttered backgrounds, varying illumination, and significant occlusions. While deep learning has advanced the field, standard CNNs often fail to distinguish between relevant human features and distracting background information, leading to false positives.

Figure 4: A typical cluttered urban scene where human detection is challenging due to busy backgrounds and varying poses.

The ViS-HuD framework addresses this by mimicking the human visual system's ability to focus on salient regions. By integrating a saliency-based pre-filtering stage, the system directs the detector's attention to areas most likely to contain humans, simplifying the feature learning process.

Methodology

The ViS-HuD pipeline consists of three primary stages:

  1. Saliency Map Generation: A Multi-Level Network (ML-Net), pre-trained on the SALICON dataset, generates a grayscale saliency map. High-intensity pixels in this map indicate regions of high visual interest.
  2. Image Masking: The original input image is multiplied element-wise by the saliency map (scaled by a factor of 0.8). This "salience-windowing" suppresses background noise while preserving the context necessary for accurate detection.
  3. Detection with DetectNet: The masked image is fed into DetectNet, a Fully Convolutional Network based on GoogLeNet. DetectNet predicts human coverage and bounding box coordinates, which are then refined through a clustering function.

Experimental Setup

The proposed method was evaluated on two challenging benchmarks:

  • Penn-Fudan Dataset: Contains 170 images with 345 labeled pedestrians in urban settings.
  • TUD-Brussels Benchmark: A larger dataset with over 4,800 images, featuring complex far-field viewpoints and heavy occlusions.
Figure 6: Examples of diverse pedestrian scenarios from the datasets used to evaluate ViS-HuD.

Results and Discussion

ViS-HuD demonstrated superior performance over traditional and modern baselines. On the Penn-Fudan dataset, it achieved a detection accuracy of 91.4%, surpassing YOLO (88.73%) and Fast R-CNN (90.38%). On the more difficult TUD-Brussels benchmark, it achieved a competitive average miss-rate of 53%, a 7% improvement over the baseline.

Key insights from the study include:

  • Saliency as a Direct Filter: Using saliency as a "soft mask" rather than just a parallel feature significantly improves robustness to occlusion.
  • Context Preservation: Scaling the saliency map (0.8 factor) prevents the creation of abrupt edges that could confuse CNN filters, maintaining enough background context for effective training.

Conclusion

ViS-HuD successfully leverages visual saliency to enhance human detection in complex environments. While the additional saliency generation step introduces some computational overhead, the gains in accuracy and robustness are substantial. Future work will focus on optimizing the pipeline for real-time applications and extending the method to temporal saliency in video tracking.

Summary updated

BibTeX
@inproceedings{gajjar2018vis,
  author = {Vandit Gajjar and Yash Khandhediya and Ayesha Gurnani and Viraj Mavani and Mehul S. Raval},
  title = {{ViS-HuD: Using Visual Saliency to Improve Human Detection With Convolutional Neural Networks}},
  booktitle = {Workshops @ CVPR},
  year = {2018},
  doi = {10.1109/cvprw.2018.00256},
}
Figure