Toward Robust Pedestrian Detection With Data Augmentation

Sebastian Cygert, Andrzej CzyzewskiIEEE Access 2020 · OpenReview

Abstract

In this article, the problem of creating a safe pedestrian detection model that can operate in the real world is tackled. While recent advances have led to significantly improved detection accuracy on various benchmarks, existing deep learning models are vulnerable to invisible to the human eye changes in the input image which raises concerns about its safety. A popular and simple technique for improving robustness is using data augmentation. In this work, the robustness of existing data augmentation techniques is evaluated to propose a new simple augmentation scheme where during training, an image is combined with a patch of a stylized version of that image. Evaluation of pedestrian detection models robustness and uncertainty calibration under naturally occurring corruption and in realistic cross-dataset evaluation setting is conducted to show that our proposed solution improves upon previous work. In this paper, the importance of testing the robustness of recognition models is emphasized and it shows a simple way to improve it, which is a step towards creating robust pedestrian and object detection models.

Introduction

Deep learning models for pedestrian detection have achieved remarkable accuracy on standard benchmarks. However, these models often possess a "glass jaw": they are notoriously fragile when faced with distributional shifts—changes in input data such as heavy fog, sensor noise, or the transition from day to night. Furthermore, these models frequently suffer from overconfidence, predicting incorrect pedestrian locations with high certainty. In safety-critical applications like autonomous driving, a model that cannot identify its own confusion is a significant liability. This research aims to bridge the gap between benchmark performance and real-world reliability.

Methodology: Patch-based Style Augmentation

The core contribution of this work is a novel data augmentation strategy designed to balance clean-data performance with robustness. While previous research suggested using global Style Transfer (applying artistic textures to training images) to force models to focus on shapes rather than pixel-level textures, this often degrades accuracy on "clean" (normal) images.

To address this, the authors propose Patch-based Style Augmentation:

  1. Stylization: A version of the training image is generated using a style-transfer algorithm with textures from the Painter by Numbers dataset.
  2. Patching: Instead of using the entire stylized image, a random patch is cut from it and overlaid onto the original clean image.
  3. Hybridization: This is often combined with Patch Gaussian augmentation, which adds noise only to specific regions.
Comparison of data augmentation techniques: Original, CutOut, Stylized, and the proposed Patch-based Style Augmentation (Our).

By stylizing only a portion of the image, the model learns to remain robust to texture distortions without losing its ability to process high-quality, real-world inputs. The authors evaluated this using two architectures: the classic Faster R-CNN and the modern, anchor-free Center-Scale-Prediction (CSP).

An example of the proposed augmentation scheme where a stylized patch is applied over a pedestrian group to force the model to rely on shape rather than texture.

Experimental Setup and Datasets

The study utilizes several key benchmarks to test different facets of robustness:

  • CityPersons: The primary "clean" training set.
  • Common Corruptions (ImageNet-C style): A benchmark of 15 synthetic distortions (e.g., blur, snow, JPEG compression) applied at five levels of severity.
  • EuroCity Persons (ECP): Used for cross-dataset evaluation, specifically comparing performance on daytime vs. night-time images.
  • NightOwls: A challenging dataset focused exclusively on night-time pedestrian detection.

Key Insights

  • The "Sweet Spot" for Patches: The size of the stylized patch is critical. A patch size of roughly 0.3 times the image dimensions provided the best balance. Smaller patches failed to induce robustness, while larger patches degraded performance on clean data.
  • Texture Bias vs. Shape Bias: CNNs tend to rely on local textures. By injecting stylized patches, the authors force the network to look for more global, robust features—like the human silhouette—that persist despite environmental noise.
  • Uncertainty Calibration: Improving accuracy does not automatically improve Uncertainty Calibration. Using the Expected Calibration Error (ECE) metric, the authors demonstrated that their augmentation helps models produce confidence scores that more accurately reflect their true likelihood of being correct.

Results

The primary metric used is LAMR (Log-Average Miss Rate), where lower scores indicate better performance.

  • Robustness to Corruptions: On the Common Corruptions benchmark, the proposed method (Our + PatchGaussian) achieved a LAMR of 42.73, significantly outperforming the baseline of 51.78.
  • Night-time Performance: When training on daytime data and testing on the ECP night-time dataset, the baseline miss rate was 45.5%. The proposed augmentation slashed this to 26.13%.
  • Calibration Consistency: The proposed model maintained consistent calibration error across different datasets, whereas the baseline model's confidence became wildly unreliable in night-time settings.

Conclusion

While the proposed method significantly improves robustness, a gap remains between "clean" and "corrupted" performance, particularly in extreme noise scenarios like the NightOwls dataset. Future work may combine patch-based augmentation with self-supervised learning or adversarial training to further harden models against the unpredictable nature of real-world environments.

Summary updated

Abstract

In this article, the problem of creating a safe pedestrian detection model that can operate in the real world is tackled. While recent advances have led to significantly improved detection accuracy on various benchmarks, existing deep learning models are vulnerable to invisible to the human eye changes in the input image which raises concerns about its safety. A popular and simple technique for improving robustness is using data augmentation. In this work, the robustness of existing data augmentation techniques is evaluated to propose a new simple augmentation scheme where during training, an image is combined with a patch of a stylized version of that image. Evaluation of pedestrian detection models robustness and uncertainty calibration under naturally occurring corruption and in realistic cross-dataset evaluation setting is conducted to show that our proposed solution improves upon previous work. In this paper, the importance of testing the robustness of recognition models is emphasized and it shows a simple way to improve it, which is a step towards creating robust pedestrian and object detection models.
BibTeX
@inproceedings{cygert2020toward,
  author = {Sebastian Cygert and Andrzej Czyzewski},
  title = {{Toward Robust Pedestrian Detection With Data Augmentation}},
  booktitle = {IEEE Access},
  year = {2020},
  url = {https://openreview.net/forum?id=WLbvOk6Xjv},
}
Figure