Adaptive Pattern-Parameter Matching for Robust Pedestrian Detection

Mengyin Liu, Chao Zhu, Jun Wang, Xu-Cheng YinAAAI 2021 · OpenReview

Abstract

Pedestrians with challenging patterns, e.g. small scale or heavy occlusion, appear frequently in practical applications like autonomous driving, which remains tremendous obstacle to higher robustness of detectors. Although plenty of previous works have been dedicated to these problems, properly matching patterns of pedestrian and parameters of detector, i.e., constructing a detector with proper parameter sizes for certain pedestrian patterns of different complexity, has been seldom investigated intensively. Pedestrian instances are usually handled equally with the same amount of parameters, which in our opinion is inadequate for those with more difficult patterns and leads to unsatisfactory performance. Thus, we propose in this paper a novel detection approach via adaptive pattern-parameter matching. The input pedestrian patterns, especially the complex ones, are first disentangled into simpler patterns for detection head by Pattern Disentangling Module (PDM) with various receptive fields. Then, Gating Feature Filtering Module (GFFM) dynamically decides the spatial positions where the patterns are still not simple enough and need further disentanglement by the next-level PDM. Cooperating with these two key components, our approach can adaptively select the best matched parameter size for the input patterns according to their complexity. Moreover, to further explore the relationship between parameter sizes and their performance on the corresponding patterns, two parameter selection policies are designed: 1) extending parameter size to maximum, aiming at more difficult patterns for different occlusion types; 2) specializing parameter size by group division, aiming at complex patterns for scale variations. Extensive experiments on two popular benchmarks, Caltech and CityPersons, show that our proposed method achieves superior performance compared with other state-of-the-art methods on subsets of different scales and occlusion types.

Introduction

Pedestrian detection is a cornerstone of autonomous driving and surveillance, but it faces two persistent "hard patterns": small scales and heavy occlusions. Most current pedestrian detectors treat every person in an image the same way, regardless of whether they are clearly visible or a blurry, occluded mess. This paper identifies a fundamental flaw in current state-of-the-art detectors: they use a fixed amount of parameters for every instance.

In reality, a large, clear pedestrian is "painless" to detect and requires fewer parameters. Conversely, a small-scale pedestrian—often just a few blurry, noisy pixels—requires more parameters to distinguish them from background clutter. By treating all patterns equally, current models under-perform on the difficult cases and over-compute on the easy ones.

Figure 1: Illustration of pedestrian detection challenges. (a) Small-scale pedestrians often appear as blurry, low-resolution patterns. (b) Heavy occlusions significantly increase the complexity of detection.

Methodology

The authors propose the Adaptive Pattern-Parameter Matching (AP$^2$M) network, an anchor-free detector built on three core innovations designed to match computational complexity to pattern difficulty.

Pattern Disentangling Module (PDM)

Instead of processing a feature map as one block, the PDM uses parallel branches with different receptive fields (via dilated convolutions). This "disentangles" complex patterns into simpler ones that the detection head can easily digest. By breaking a complex scene into parallel, scale-specific representations, the final detection head has a much easier job.

Gating Feature Filtering Module (GFFM)

This acts as a smart traffic controller. It generates a spatial heatmap to decide which parts of an image are "simple enough" and which parts are still "too complex." The complex regions are passed to the next level of PDM for further processing, while simple regions are filtered out to prevent overfitting. This mechanism allows the network to effectively become deeper only where it needs to be.

Specialized Parameter Policies

The authors designed two specific versions of the PDM to handle different types of complexity:

  • Extended PDM (E-PDM): Increases parameter size specifically to handle the ambiguity of different occlusion types.
  • Grouped PDM (G-PDM): Uses group convolutions to provide more parameters for smaller receptive fields, specifically targeting small-scale pedestrians.

Experimental Results

The model was evaluated on two industry-standard benchmarks: the Caltech Pedestrian Dataset, known for its high percentage of small-scale pedestrians, and CityPersons, which features diverse occlusion levels.

AP$^2$M achieved state-of-the-art results on both major benchmarks, particularly in the "Hard" categories:

  • Caltech: Achieved a Log Miss Rate ($MR^{-2}$) of 3.3% on the "Reasonable" subset, outperforming the previous best (OR-CNN at 4.1%). On the "Heavy Occlusion" subset, it reached 42.2%, a significant improvement over the baseline CSP (45.8%).
  • CityPersons: Achieved 10.4% $MR^{-2}$ on the "Reasonable" subset. It showed its greatest strength in the "Small" pedestrian category, hitting 15.3% compared to the baseline's 16.0%.

Conclusion

The core insight of AP$^2$M is that small, blurry objects actually need more "model brainpower" (parameters) than large, clear ones. By adaptively matching parameter size to pattern complexity, the framework significantly improves robustness. While the paper demonstrates superior accuracy, future work could involve optimizing the gating mechanism to prune unnecessary branches more aggressively, potentially creating a model that is not only more robust but also faster by dynamically skipping computations for "easy" frames.

Summary updated

Abstract

Pedestrians with challenging patterns, e.g. small scale or heavy occlusion, appear frequently in practical applications like autonomous driving, which remains tremendous obstacle to higher robustness of detectors. Although plenty of previous works have been dedicated to these problems, properly matching patterns of pedestrian and parameters of detector, i.e., constructing a detector with proper parameter sizes for certain pedestrian patterns of different complexity, has been seldom investigated intensively. Pedestrian instances are usually handled equally with the same amount of parameters, which in our opinion is inadequate for those with more difficult patterns and leads to unsatisfactory performance. Thus, we propose in this paper a novel detection approach via adaptive pattern-parameter matching. The input pedestrian patterns, especially the complex ones, are first disentangled into simpler patterns for detection head by Pattern Disentangling Module (PDM) with various receptive fields. Then, Gating Feature Filtering Module (GFFM) dynamically decides the spatial positions where the patterns are still not simple enough and need further disentanglement by the next-level PDM. Cooperating with these two key components, our approach can adaptively select the best matched parameter size for the input patterns according to their complexity. Moreover, to further explore the relationship between parameter sizes and their performance on the corresponding patterns, two parameter selection policies are designed: 1) extending parameter size to maximum, aiming at more difficult patterns for different occlusion types; 2) specializing parameter size by group division, aiming at complex patterns for scale variations. Extensive experiments on two popular benchmarks, Caltech and CityPersons, show that our proposed method achieves superior performance compared with other state-of-the-art methods on subsets of different scales and occlusion types.
BibTeX
@inproceedings{liu2021adaptive,
  author = {Mengyin Liu and Chao Zhu and Jun Wang and Xu-Cheng Yin},
  title = {{Adaptive Pattern-Parameter Matching for Robust Pedestrian Detection}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2021},
  url = {https://openreview.net/forum?id=ixAMPfIuZ1},
}
Figure