Compensation Tracker: Reprocessing Lost Object for Multi-Object Tracking

Zhibo Zou, Junjie Huang, Ping LuoWACV 2022

TL;DR

This paper introduces a "Compensation Tracker" (CT) that effectively re-tracks objects lost due to detector failures in multi-object tracking (MOT) systems. By leveraging motion prediction and a novel object selection module, CT improves tracking performance and reduces identity switches without requiring additional complex networks or retraining, making it suitable for real-time applications.

What Problem Does This Paper Address?

Multi-object tracking (MOT) is crucial for applications like autonomous driving and surveillance. The dominant "tracking-by-detection" (TBD) paradigm relies heavily on the performance of object detectors. However, these detectors can sometimes fail to detect objects in certain frames, leading to "lost objects." When an object is lost, it's typically dropped from the tracking results, causing a loss of identity and potentially leading to identity switches. Existing methods to address this often involve complex, computationally expensive additional networks, which are not ideal for real-time systems. This paper tackles the challenge of robustly re-acquiring and re-identifying these lost objects in a computationally efficient manner, particularly in challenging dense crowd scenarios.

Methods

The proposed Compensation Tracker (CT) is designed as an extension to existing TBD trackers. It consists of two main modules:

  • Motion Compensation (MC): This module uses a Kalman filter to predict the position of lost objects based on their past motion. This prediction helps to re-establish the object's presence even if the detector missed it.
  • Object Selection (OS): This is the core novelty. It filters the predicted bounding boxes from the MC module to identify highly credible lost objects that are likely to be actual missed detections rather than false positives or objects that have genuinely disappeared. The OS module employs several filtering steps:
  • Confidence Interference Filtering (CI_Filtering): Filters based on the object's tracking history (number of times tracked vs. lost).
  • Boundary Interference Filtering (BI_Filtering): Ensures the predicted object remains within a reasonable area relative to the image boundaries.
  • IoU Interference Filtering (IoU_Filtering): Prevents re-tracking objects that are occluded or overlap significantly with already tracked objects.
  • Appearance Interference Filtering (AI_Filtering): Compares the appearance of the predicted bounding box with the appearance of the object in the previous frame using hand-crafted features (HOG) and KNN matching.

If an object passes these selection criteria, it's considered a "missing tracking object" and is re-integrated into the tracking results. The CT module can be plugged into existing trackers without retraining.

Datasets

The paper evaluates its method on the following datasets:

  • MOT2015
  • MOT2016
  • MOT2017
  • MOT2020 (training and test datasets)

The authors specifically analyze the phenomenon of lost objects on the MOT2020 dataset.

Key Insights and Ideas

  • The "lost object" problem is significant: The paper highlights that even state-of-the-art trackers struggle with objects that are temporarily missed by detectors, leading to performance degradation.
  • Motion prediction alone isn't enough: While motion compensation helps re-predict positions, it can lead to false positives if not combined with robust selection criteria.
  • A multi-stage filtering approach is effective: The combination of confidence, boundary, IoU, and appearance filtering effectively identifies reliable re-detections from predicted positions.
  • Leveraging traditional methods for efficiency: By using hand-crafted features and Kalman filters, CT achieves its compensation goals without the computational overhead of deep learning-based re-identification networks, making it suitable for real-time use.
  • Plug-and-play capability: The ability to integrate CT into existing trackers without retraining is a significant practical advantage.

Key Results

  • On the MOT2020 dataset, the Compensation Tracker (CT) significantly improved baseline tracking performance.
  • CT achieved 66% MOTA and 67% IDF1 on the MOT2020 dataset.
  • The method successfully enhanced tracking performance by a large margin with limited computational cost, adding only approximately 0.6 FPS for motion compensation and 1.1 FPS for object selection.
  • Experiments showed that CT effectively reduces identity switches, particularly in dense crowd scenarios.
  • Ablation studies demonstrated that both motion compensation and object selection contribute to the overall performance improvement, with the object selection module being crucial for filtering out false positives.

Limitations and Future Work

  • Dependence on detector quality: While CT mitigates the impact of missed detections, its performance is still indirectly influenced by the detector's overall accuracy.
  • Hand-crafted features: The reliance on hand-crafted features (like HOG) for appearance matching might limit its adaptability to highly diverse or challenging appearance changes compared to learned features.
  • Future work could explore:
    • Integrating learned appearance features for more robust re-identification.
  • Further optimizing the filtering stages for even greater efficiency.
  • Investigating the application of CT to other tracking-by-detection frameworks and datasets.

Summary updated

BibTeX
@inproceedings{zou2022compensation,
  author = {Zhibo Zou and Junjie Huang and Ping Luo},
  title = {{Compensation Tracker: Reprocessing Lost Object for Multi-Object Tracking}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2022},
}
Figure