A fast multi-object tracking system using an object detector ensemble

Richard Cobos, Jefferson Hernandez, Andres G. AbadarXiv 2019 · DOI · arXiv · OpenReview

TL;DR

This paper tackles the computational bottleneck in real-time multi-object tracking (MOT) by proposing a novel system that uses an ensemble of object detectors, each running at a reduced frame rate. This approach achieves significantly faster processing speeds than existing methods on the MOT16 benchmark, while maintaining competitive accuracy, making real-time MOT applications more feasible.

What Problem Does This Paper Address?

Multiple-Object Tracking (MOT) is essential for applications like video surveillance and retail analytics. A common MOT approach is "tracking-by-detection," which involves two steps: detecting objects and then associating these detections to form trajectories. The primary challenge in this paradigm is the computational cost of object detection, which often runs on every frame, making real-time tracking difficult. This paper aims to overcome this by reducing the computational load of the detection stage without sacrificing too much accuracy.

Methods

The core innovation lies in an object detection ensemble that runs every f frames, rather than every frame. This ensemble combines predictions from multiple object detectors. To reconcile detections from different detectors and handle overlapping bounding boxes, they adapt the Soft-NMS (Non-Maximum Suppression) algorithm. The tracking itself uses a Kalman filter-based approach, similar to existing methods, but benefits from the reduced detection load. The system models object states and transitions, and uses a cost-based assignment problem to match detections to existing tracks. A key aspect is the use of a statistical distance measure that accounts for uncertainty, improving robustness.

Datasets

The paper evaluates its system on the MOT16 benchmark, a standard dataset for multi-object tracking. No new datasets were introduced.

Key Insights and Ideas

  • Reduced Detection Frequency is Key: Running object detectors less frequently (every f frames) is a viable strategy to achieve real-time performance without a drastic drop in accuracy.
  • Ensemble Power: Combining multiple detectors, even if run less often, can improve overall detection robustness and reduce false negatives compared to a single detector.
  • Soft-NMS Adaptation: Adapting Soft-NMS for combining ensemble detections is an effective way to merge and score overlapping bounding boxes from different models.
  • Trade-off Management: The paper explicitly demonstrates the trade-off between frame skipping (f) and accuracy, showing that a moderate skip rate (f=5) offers a good balance.

Key Results

The proposed system achieved the highest speed among online entries on the MOT16 challenge, running at 249.6 Hz (when considering only the tracker, not the detector ensemble). When the ensemble detector was included, the system still achieved real-time performance. For instance, with f=5, the system achieved a competitive MOTA score while significantly outperforming other methods in speed. The ensemble detector itself showed an average accuracy improvement of 7.8% (MOTA) over the best individual detector when f=1, and 5.5% when f=5.

Limitations and Future Work

The authors acknowledge that skipping frames (f > 1) leads to a degradation in accuracy. They also note that their system, while fast, does not incorporate person re-identification, which could further improve tracking robustness, especially in cases of long-term occlusion. Future work directions include:

  • Improving the persistence of tracked objects.
  • Reducing reliance on perfect object detectors.
  • Incorporating more object detectors into the ensemble.
  • Adding person re-identification capabilities.

Summary updated

Abstract

Multiple-Object Tracking (MOT) is of crucial importance for applications such as retail video analytics and video surveillance. Object detectors are often the computational bottleneck of modern MOT systems, limiting their use for real-time applications. In this paper, we address this issue by leveraging on an ensemble of detectors, each running every f frames. We measured the performance of our system in the MOT16 benchmark. The proposed model surpassed other online entries of the MOT16 challenge in speed, while maintaining an acceptable accuracy.
BibTeX
@misc{cobos2019fast,
  author = {Richard Cobos and Jefferson Hernandez and Andres G. Abad},
  title = {{A fast multi-object tracking system using an object detector ensemble}},
  year = {2019},
  eprint = {1908.04349},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1908.04349},
}
Figure