ConfTrack: Kalman Filter-Based Multi-Person Tracking by Utilizing Confidence Score of Detection Box

Hyeonchul Jung, Seokjun Kang, Takgen Kim, HyeongKi KimWACV 2024

TL;DR

This paper introduces ConfTrack, a novel Kalman Filter-based tracking-by-detection (KFTBD) algorithm designed to improve multi-person tracking in challenging, crowded scenarios. By intelligently handling low-confidence object detections, ConfTrack significantly reduces ID switches and tracking failures, achieving state-of-the-art results on the MOT20 dataset.

What Problem Does This Paper Address?

Multi-person tracking (MPT) is crucial for applications like autonomous driving and surveillance. A dominant approach is tracking-by-detection (TBD), where object detections are used to build and maintain trajectories. While Kalman Filters (KFTBD) are efficient for this, they struggle in crowded environments where object detectors often produce noisy bounding boxes with low confidence scores. These unreliable detections can lead to incorrect Kalman filter predictions, destabilize track states, and ultimately cause ID switches or complete tracking failures. Existing methods often discard low-confidence detections, missing potentially valuable information.

Methods

ConfTrack builds upon the KFTBD framework, specifically the BoT-SORT baseline, and introduces several novel components to address the challenge of low-confidence detections:

  • Confidence Weighted Kalman-Update (CW): Instead of directly using a low-confidence detection box in the Kalman update, ConfTrack modifies the target measurement. It blends the original detection box with the Kalman filter's prediction, weighted by the detection's confidence score. This makes the update more robust to noisy detections.
  • Noise Scale Adaptive Kalman Filter (NK): ConfTrack adapts the Noise Scale Adaptive Kalman Filter (NSAK) by introducing an amplifying factor. This allows the filter to amplify the measurement space noise covariance for low-confidence detections, effectively reducing the Kalman gain and making the filter less reliant on potentially inaccurate measurements.
  • Constant Box Prediction on Lost Track (CP): To mitigate unstable box size variations from noisy detections when a track is lost, ConfTrack sets the width and height of the predicted box to zero. This helps the Kalman filter maintain a more stable prediction for lost tracks.
  • Confidence Fused Cost Matrix (CF): This method enhances the cost matrix used for matching tracks and detections by incorporating detection confidence scores. It's applied to both IoU-based and ReID feature-based costs, making the association more robust.
  • Cascade Matching with Low-Confidence Track (LM): This is a key algorithmic contribution. ConfTrack employs a two-stage matching process. The first stage prioritizes high-confidence tracks and detections. The second stage then intelligently matches remaining tentative tracks (including those initialized from low-confidence detections) with remaining detections. This cascade approach prevents low-confidence tracks from incorrectly matching with low-confidence detections, thus avoiding the initialization of wrong confirmed tracks.

Datasets

ConfTrack was evaluated on several diverse datasets:

  • MOT17 & MOT20: Standard benchmarks for multi-person tracking, with MOT20 specifically focusing on more crowded environments.
  • DanceTrack: Features videos of people dancing, characterized by non-linear motion.
  • HiEve: A large-scale dataset with variations in resolution, camera angles, and actions.

The paper also mentions using weights learned on the COCO dataset for detectors.

Key Insights and Ideas

  • Low-confidence detections are not just noise; they contain useful information. Instead of discarding them, ConfTrack intelligently integrates them, leading to better performance.
  • The Kalman filter's prediction is often more reliable than a low-confidence detection. This insight drives the confidence-weighted update and noise scaling mechanisms.
  • A cascaded matching strategy is crucial for handling low-confidence data. Separating matching stages and prioritizing high-confidence associations first prevents cascading errors from low-confidence initializations.
  • Robustness to noise is paramount for crowded scenarios. ConfTrack's success on MOT20 highlights its effectiveness in environments where detection quality degrades significantly.

Key Results

ConfTrack achieved state-of-the-art performance on the MOT20 dataset, securing the highest scores for HOTA and IDF1 metrics. Ablation studies demonstrated that each novel component (CW, NK, CP, CF, LM) contributed positively to the overall performance, with NK showing a particularly strong impact. The method also showed good compatibility with different detectors (YOLOX, YOLOv7, Transformer-based DINO), indicating its general applicability.

Limitations and Future Work

The authors acknowledge that KFTBD-based trackers, including ConfTrack, are designed based on linear models like the Kalman filter. This inherently poses limitations for tracking highly non-linear object motions, as illustrated by failure cases in the DanceTrack dataset. Future work could explore incorporating more advanced non-linear motion modeling techniques to further enhance tracking performance in such challenging scenarios.

Summary updated

BibTeX
@inproceedings{jung2024conftrack,
  author = {Hyeonchul Jung and Seokjun Kang and Takgen Kim and HyeongKi Kim},
  title = {{ConfTrack: Kalman Filter-Based Multi-Person Tracking by Utilizing Confidence Score of Detection Box}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2024},
}
Figure