Distant Pedestrian Detection in the Wild using Single Shot Detector with Deep Convolutional Generative Adversarial Networks

Ranjith Dinakaran, Philip Easom, Li Zhang, Ahmed Bouridane, Richard Jiang, Eran A. Edirisinghe

TL;DR

This paper proposes a novel approach to improve pedestrian detection in challenging "in-the-wild" scenarios by combining Deep Convolutional Generative Adversarial Networks (DCGANs) with the Single Shot Detector (SSD). By using DCGANs to generate realistic variations of low-resolution images, the authors effectively augment training data, leading to significantly better detection performance, especially for distant pedestrians.

What Problem Does This Paper Address?

Detecting pedestrians in real-world scenarios, often referred to as "in-the-wild," presents a significant challenge for computer vision systems. This is due to factors like varying lighting conditions, occlusions, and importantly, the difficulty in obtaining sufficient training data, especially for distant or small pedestrians. Traditional object detection methods often struggle with these limitations, leading to missed detections. The paper specifically targets the problem of insufficient and imbalanced training data for pedestrian detection, particularly for small and distant objects, which are crucial for applications like smart city surveillance.

Methods

The core of this work lies in the synergistic combination of DCGANs and SSD.

  • DCGAN for Data Augmentation: The authors employ DCGANs to address the data scarcity problem. Specifically, they use an "in-fill completion" technique where portions of low-resolution images are masked and then filled in by the GAN. This process generates diverse, realistic variations of existing images, effectively expanding the training dataset without requiring manual annotation of new data. The GAN is trained intensively on low-resolution images to learn to generate high-quality enhancements.
  • Single Shot Detector (SSD): SSD is a popular and efficient object detection framework that uses a single feed-forward convolutional network to predict bounding boxes and class scores. It's known for its speed and ability to detect objects at various scales.
  • DCGAN-SSD Architecture: The proposed architecture cascades the DCGAN with SSD. The DCGAN acts as a pre-processing step, enhancing low-resolution images to a higher resolution and filling in missing contextual information. These enhanced images are then fed into the SSD for the actual pedestrian detection task. This approach aims to leverage the generative capabilities of DCGANs to improve the input quality for the discriminative SSD model.

Datasets

The paper utilizes the following datasets for training and evaluation:

  • CIFAR-10: A widely used dataset for image classification and object detection. The authors specifically mention using it for training and testing under different resolutions.
  • Caltech: Another dataset commonly used for object detection tasks.
  • KITTI: A dataset known for its focus on autonomous driving scenarios, which includes pedestrian detection.
  • VOC (PASCAL VOC): A benchmark for object detection and segmentation tasks.

The authors do not introduce any new benchmarks but rather focus on improving performance on existing ones, particularly for the challenge of distant pedestrian detection.

Key Insights and Ideas

  • GANs as Powerful Data Augmenters: The paper demonstrates that GANs, specifically DCGANs, can be highly effective in generating realistic synthetic data to overcome the limitations of small or imbalanced datasets in object detection.
  • Enhancing Low-Resolution Images for Detection: By training GANs on low-resolution images and using them to generate higher-resolution, contextually rich versions, the authors show a significant improvement in the ability to detect small and distant objects.
  • Synergy of Generative and Discriminative Models: The successful combination of a generative model (DCGAN) with a discriminative model (SSD) highlights a promising direction for tackling complex computer vision problems where data scarcity is a bottleneck.
  • Improved Performance on Challenging Scenarios: The proposed DCGAN-SSD architecture shows a notable improvement in detecting pedestrians that are often missed by standard SSD, particularly those further away from the camera.

Key Results

The paper reports substantial improvements in pedestrian detection accuracy when using the DCGAN-SSD architecture compared to using SSD alone.

  • Quantitative Results: Table 1 shows a significant increase in performance across various datasets. For instance, on the CIFAR dataset, the DCGAN+SSD method achieves an F1 score of 87.9%, while SSD alone achieves 53.5%. Similar improvements are observed for Caltech, KITTI, and VOC datasets. Figure 6 visually demonstrates this improvement, showing that DCGAN+SSD successfully detects objects missed by SSD alone. Figure 7 presents precision-recall curves, further illustrating the superior performance of the combined approach, especially in detecting distant objects.
  • Qualitative Findings: The enhanced images produced by the DCGAN are visually more detailed and contextually complete, which directly aids the SSD in identifying and localizing pedestrians more effectively.

Limitations and Future Work

  • Resolution Limitations: The authors encountered difficulties in training GAN models on images with resolutions larger than 64x64, indicating that achieving convergence for very high-resolution images might require further architectural or training modifications.
  • Domain Robustness: While the DCGAN was not explicitly trained on CIFAR-10, the experiments suggest domain robustness of the learned features, which is a positive sign.
  • Future Work: The paper suggests that further improvements could be made by fine-tuning the discriminator's representations. Additionally, the authors mention that the resulting models could be deployed for other computer vision tasks like image description generation, semantic image segmentation, and facial/gesture expression recognition.

Summary updated

Abstract

In this work, we examine the feasibility of applying Deep Convolutional Generative Adversarial Networks (DCGANs) with Single Shot Detector (SSD) as data-processing technique to handle with the challenge of pedestrian detection in the wild. Specifically, we attempted to use in-fill completion to generate random transformations of images with missing pixels to expand existing labelled datasets. In our work, GAN’s been trained intensively on low resolution images, in order to neutralize the challenges of the pedestrian detection in the wild, and considered humans, and few other classes for detection in smart cities. The object detector experiment performed by training GAN model along with SSD provided a substantial improvement in the results. This approach presents a very interesting overview in the current state of art on GAN networks for object detection. We used Canadian Institute for Advanced Research (CIFAR), Caltech, KITTI data set for training and testing the network under different resolutions and the experimental results with comparison been showed between DCGAN cascaded with SSD and SSD itself.
BibTeX
@inproceedings{dinakaran2019distant,
  author = {Ranjith Dinakaran and Philip Easom and Li Zhang and Ahmed Bouridane and Richard Jiang and Eran A. Edirisinghe},
  title = {{Distant Pedestrian Detection in the Wild using Single Shot Detector with Deep Convolutional Generative Adversarial Networks}},
  booktitle = {IJCNN},
  year = {2019},
  url = {https://openreview.net/forum?id=gnKsvtyCwC},
}
Figure