Tracking Algorithm of Multiple Pedestrians Based on Particle Filters in Video Sequences

Hui Li, Yun Liu, Chuanxu Wang, Shujun Zhang, Xuehong CuiComputational 2016 · DOI

TL;DR

This paper introduces a robust particle filter-based algorithm for tracking multiple pedestrians in complex video sequences. It tackles challenges like occlusion and changing backgrounds by integrating prior knowledge for detection, using multi-feature observation models, and dynamically adjusting feature weights. The method demonstrates improved tracking accuracy and robustness compared to existing approaches.

What Problem Does This Paper Address?

Tracking multiple pedestrians in real-world video is a notoriously difficult problem in computer vision. Existing methods often struggle with:

  • Dynamic Environments: Moving backgrounds, changing lighting, and variations in pedestrian appearance (posture, scale).
  • Interactions: Pedestrians frequently occlude each other, making it hard to maintain individual tracks.
  • Appearance and Disappearance: Pedestrians entering or leaving the scene can disrupt tracking.
  • Computational Cost: Many advanced methods are too slow for real-time applications.

The authors aim to develop a system that can reliably track multiple individuals simultaneously, even under these challenging conditions, by leveraging the strengths of particle filters while addressing their limitations.

Methods

The proposed algorithm builds upon the particle filter framework and incorporates several novel components:

  • Prior Knowledge Extraction for Detection: Instead of relying solely on detection in each frame, the algorithm first extracts "prior knowledge" from the initial frames. This involves segmenting the scene into object and background regions and using superpixel analysis to learn characteristic features. This prior knowledge helps in more robust initial detection and in distinguishing between object and background elements.
  • Confidence Map Generation: Based on the prior knowledge, a confidence map is generated for each frame, highlighting areas likely to contain pedestrians. This map is then used for sampling candidate regions for tracking.
  • Multi-Feature Observation Model: The particle filter's observation model utilizes a combination of HSV color histograms and Local Binary Pattern (LBP) histograms. This multi-feature approach provides a richer representation of the pedestrian's appearance, making it more resilient to minor changes.
  • Adaptive Feature Weighting: Crucially, the algorithm dynamically adjusts the weights of the color and LBP features based on the current tracking environment. This allows the system to prioritize the more informative feature under different conditions (e.g., relying more on texture if color is unreliable due to lighting).
  • Occlusion Handling and Object State Management: The paper explicitly addresses severe occlusion by maintaining particle states even when an object is temporarily lost. It also incorporates a method to detect when objects disappear or appear, allowing for the removal of lost tracks and the initialization of new ones, preventing drift and maintaining a consistent set of tracked individuals.

Datasets

The authors evaluated their algorithm on several datasets:

  • Custom Database: Their own collection of video sequences.
  • CAVIAR Database: A well-known dataset for pedestrian detection and tracking.
  • PETS2012 Standard Library: A benchmark dataset for pedestrian detection and tracking, often used for evaluating performance in complex scenarios.

The paper does not introduce a new benchmark dataset but rather uses existing ones to demonstrate the effectiveness of their proposed method.

Key Insights and Ideas

  • Prior Knowledge is Key for Robustness: Leveraging information from initial frames to build a model of the object and background significantly improves initial detection and subsequent tracking, especially in dynamic scenes.
  • Multi-Feature Fusion with Adaptive Weights: Combining complementary features (color and texture) is powerful, but dynamically adjusting their importance based on the environment is essential for handling varying conditions and preventing tracking failures.
  • Distinguishing Appearance/Disappearance is Crucial for Multi-Object Tracking: Explicitly modeling and detecting when objects enter or leave the scene is vital for managing multiple tracks and avoiding confusion.
  • Particle Filters as a Flexible Framework: The paper highlights the adaptability of particle filters for non-linear, non-Gaussian problems like pedestrian tracking, especially when augmented with sophisticated observation models and state management.

Key Results

The proposed algorithm demonstrated improved performance across several video sequences compared to two established methods: superpixel tracking and BPF (Bayesian Particle Filter) tracking.

  • Reduced Error: The average root mean square error (RMSE) was consistently lower for the proposed algorithm across various pedestrians and sequences. For instance, in Sequence 1, Pedestrian 3 achieved an RMSE of 7.3893, significantly better than superpixel (29.447) and BPF (6.187) for Pedestrian 2. (Note: There seems to be a slight discrepancy in the text regarding which pedestrian had which score, but the trend of lower error for the proposed method is clear).
  • Robustness to Background Changes: The algorithm showed resilience to dynamic backgrounds, unlike the compared methods which exhibited tracking drift.
  • Effective Occlusion Handling: The method successfully handled severe occlusion, maintaining tracks and recovering them when objects reappeared.
  • Accurate Appearance/Disappearance Detection: The algorithm accurately identified when pedestrians entered or left the scene, which is critical for multi-object tracking.

Limitations and Future Work

  • Common Tracking Algorithm for Complex Scenarios: The authors acknowledge that while their algorithm is robust, there is still a need for a "common tracking algorithm for various kinds of complex environments," suggesting that further generalization is desirable.
  • Future Improvements: They mention that improvements and amendments are still needed in future studies, particularly for pedestrian tracking in moving objects. This implies that while their method is good, there's room for further refinement and expansion to handle even more challenging scenarios.

Summary updated

Abstract

Pedestrian tracking is a critical problem in the field of computer vision. Particle filters have been proven to be very useful in pedestrian tracking for nonlinear and non-Gaussian estimation problems. However, pedestrian tracking in complex environment is still facing many problems due to changes of pedestrian postures and scale, moving background, mutual occlusion, and presence of pedestrian. To surmount these difficulties, this paper presents tracking algorithm of multiple pedestrians based on particle filters in video sequences. The algorithm acquires confidence value of the object and the background through extracting a priori knowledge thus to achieve multipedestrian detection; it adopts color and texture features into particle filter to get better observation results and then automatically adjusts weight value of each feature according to current tracking environment. During the process of tracking, the algorithm processes severe occlusion condition to prevent drift and loss phenomena caused by object occlusion and associates detection results with particle state to propose discriminated method for object disappearance and emergence thus to achieve robust tracking of multiple pedestrians. Experimental verification and analysis in video sequences demonstrate that proposed algorithm improves the tracking performance and has better tracking results.
BibTeX
@inproceedings{li2016tracking,
  author = {Hui Li and Yun Liu and Chuanxu Wang and Shujun Zhang and Xuehong Cui},
  title = {{Tracking Algorithm of Multiple Pedestrians Based on Particle Filters in Video Sequences}},
  booktitle = {Computational},
  year = {2016},
  doi = {10.1155/2016/8163878},
}
Figure