Abstract
This paper provides a comprehensive survey and evaluation of modern face and person tracking algorithms specifically tailored for video surveillance. By categorizing methods into appearance and motion models and applying a Technology Readiness Level (TRL) framework, the authors identify sparse representation-based methods as the current "state-of-the-art" for handling the occlusions and appearance changes typical of high-security environments like airports.
Introduction
While general object tracking is a well-studied field in computer vision, video surveillance introduces a unique set of "non-negotiable" requirements. A surveillance tracker must operate in real-time, handle frequent occlusions (people walking behind pillars or each other), manage drastic lighting changes, and—most importantly—maintain a consistent identity for a target even after they leave and re-enter the camera's field of view.
The authors address the gap between theoretical tracking research and practical deployment. They evaluate whether current algorithms are actually ready for specific real-world scenarios: isolated people in lanes (PIL), controlled chokepoints (security gates), uncontrolled doorways, and dense outdoor crowds.
Methodology: Appearance and Motion Models
The tracking problem is decomposed into two primary components: appearance models and motion models.
Appearance Models
Appearance models define how the tracker "sees" and identifies the target. The paper categorizes these into several types:
- Template Matching: Simple but prone to "drift" when a person turns their head.
- Histograms: Robust to motion blur but lose spatial information. Fragtrack is highlighted for using part-based histograms to handle partial occlusions.
- Local Features: Keypoints like SIFT, SURF, or binary descriptors (BRIEF/BRISK) are excellent for handling rotation and scale changes.
- Sparse Representations: A key focus of the paper. These methods use a dictionary of templates and solve an $\ell_1$-minimization problem to reconstruct the target, effectively treating occlusions as "noise."
- Learned Models: Real-time machine learning (e.g., Adaboost, Multiple Instance Learning) is used to distinguish the target from the background.
Motion Models
Motion models govern how the tracker moves the search area:
- Bayesian Filters: Specifically Particle Filters, which represent possible locations as a set of "particles" evolving over time.
- Tracking-by-Detection: Leveraging object detectors to re-seed or verify tracker results.
Key Insights and Technical Considerations
Handling Occlusion and Drift
A major challenge in tracking is the "drift" dilemma: updating templates too frequently can cause the tracker to learn the background, while updating too rarely leads to failure during pose changes. Sparse representation methods, such as the Collaborative Tracker, mitigate this by representing a face as a linear combination of previous views, allowing the system to mathematically "ignore" occluding objects like books or hands.
Computational Requirements
For particle filters to be viable in surveillance, they require thousands of particles. The paper notes that GPU acceleration is mandatory, enabling the processing of 10,000 particles in under 90ms, which is essential for real-time high-fidelity tracking.
Technology Readiness Level (TRL)
The authors use the TRL (1-9) scale to assess maturity. Most high-end trackers currently sit at TRL 4 (lab validation) or TRL 5 (relevant environment validation). None are yet considered "flight-proven" (TRL 9) for fully autonomous surveillance.
Results and Evaluation
The studied methods were evaluated across standard benchmarks like PETS-2006, CAVIAR, and TUD:
- Isolated Tracking (PIL): Distribution Fields achieved 100% accuracy on occluded face sequences with a mean error of only 5 pixels.
- Dense Traffic: In the CAVIAR dataset, the Adaptive Structural Local Sparse Tracker significantly outperformed older methods, reducing tracking error to 2.3 pixels compared to Fragtrack's 116 pixels.
- Outdoor Performance: The CRF Multi-Target Tracker successfully tracked 70% of trajectories in difficult outdoor settings, though identity switches (IDS) remain a challenge.
Conclusion and Future Work
Despite advancements, significant gaps remain. There is a lack of research and datasets for "chokepoint" scenarios, such as security checkpoints. Outdoor tracking remains the most difficult due to unpredictable lighting and crowd density. Furthermore, many high-performing trackers still require manual initialization. Future work must focus on tighter integration between automated detection and long-term tracking to achieve fully autonomous systems.
Summary updated