Computer Vision - ECCV 2016 Workshops - Amsterdam, The Netherlands, October 8-10 and 15-16, 2016, Proceedings, Part II

Gang Hua, Yiannis Demiris, Hervé JeǵouECCV 2016 · DOI · DBLP

Introduction

Multi-Object Tracking (MOT) is a fundamental task in computer vision, critical for applications ranging from autonomous driving to human-robot interaction. Despite significant progress, trackers often struggle with "track drift," where occlusions or appearance changes cause the system to lose its target. Furthermore, many existing systems are limited to specific object classes (e.g., pedestrians) and struggle with the dynamic nature of real-world scenes where objects frequently enter and exit the frame.

This paper, presented at the ECCV 2016 workshops, introduces the Multi-Class Multi-Object Tracking (MCMOT) framework. MCMOT addresses these challenges by combining deep learning-based detection with a novel Changing Point Detection (CPD) algorithm to identify and recover from tracking failures.

The MCMOT Framework

The MCMOT framework is built upon a Bayesian filtering foundation, utilizing Markov Chain Monte Carlo (MCMC) sampling for efficient state estimation. The system is designed to be class-agnostic and computationally efficient.

Overview of the MCMOT framework, illustrating the pipeline from video sequence to final trajectory combination.

The framework employs an ensemble of four detectors to compute object likelihoods:

  • Global Detector (GT): A Faster R-CNN/ResNet model for general object detection.
  • Local Detector (LT): A detector fine-tuned on-the-fly using confident track segments to reduce false negatives.
  • Color Detector (CT): Uses RGB histograms and Bhattacharyya distance to maintain appearance consistency.
  • Motion Detector (MT): A Lucas-Kanade Tracker (KLT) for position prediction.

Changing Point Detection and Validation

The core innovation of MCMOT is the Changing Point Detection (CPD) algorithm. By treating a track as a time series of data attributes, the system can detect abrupt changes that signify potential drift or occlusion.

Flowchart of the Changing Point Detection and Forward-backward Validation process used to filter track segments.

When a change point is detected, the system performs Forward-Backward (FB) validation. This involves a reverse-sequence check; if the backward track does not align with the forward track, the segment is identified as unstable and is either pruned or corrected. This proactive approach to failure detection allows the system to maintain high-quality trajectories even in cluttered environments.

Evaluation and Results

The authors evaluated MCMOT on two major benchmarks: ImageNet VID and MOT 2016.

  • ImageNet VID: MCMOT achieved a mean Average Precision (mAP) of 74.5%, outperforming the previous state-of-the-art T-CNN method (73.8%).
  • MOT 2016: The system reached a 62.4% MOTA (Multi-Object Tracking Accuracy), which was a leading score at the time of publication.

Ablation studies confirmed the impact of the new components: adding CPD to the baseline improved mAP by 6.4%, and incorporating FB validation added a further 3.4%.

Conclusion

MCMOT demonstrates that acknowledging and actively detecting tracking failures is a powerful strategy for robust multi-object tracking. By integrating deep learning detectors with CPD and FB validation, the framework achieves state-of-the-art performance across diverse object categories. While the system currently faces challenges with identity switches and fragmentation over long occlusions, it sets a strong foundation for future research in class-agnostic, resilient tracking.

Summary updated

BibTeX
@inproceedings{hua2016computer,
  author = {Gang Hua and Yiannis Demiris and Hervé Jeǵou},
  title = {{Computer Vision - ECCV 2016 Workshops - Amsterdam, The Netherlands, October 8-10 and 15-16, 2016, Proceedings, Part II}},
  booktitle = {European Conference on Computer Vision},
  year = {2016},
  doi = {10.1007/978-3-319-48881-3},
}
Figure