UCMCTrack: Multi-Object Tracking with Uniform Camera Motion Compensation

Kefu Yi, Kai Luo, Xiaolei Luo, Jiangui Huang, Hao Wu, Rongdong Hu, Wei HaoAAAI 2024 · DOI

Introduction

In the field of Multi-Object Tracking (MOT), the "tracking-by-detection" paradigm is standard but faces a significant hurdle: camera motion. When a camera shakes, pans, or tilts, objects appear to "jump" across the image plane. Traditional trackers rely on Intersection over Union (IoU) to match detections between frames. However, if the camera moves too fast or the frame rate is low, bounding boxes from consecutive frames may not overlap, causing the tracker to lose the target. While standard Camera Motion Compensation (CMC) can mitigate this, its frame-by-frame image registration is computationally heavy, often preventing real-time performance on edge devices.

UCMCTrack is a high-performance, motion-only tracker designed to overcome these challenges. By shifting tracking logic from the 2D image plane to the 3D ground plane and utilizing a novel "Mapped Mahalanobis Distance," it achieves state-of-the-art accuracy at speeds exceeding 1000 FPS.

Methods

UCMCTrack introduces several innovations to simplify motion modeling and improve robustness to camera jitter by projecting tracking data onto the ground plane.

The UCMCTrack pipeline, illustrating the transition from image plane to ground plane tracking.

The core technical components include:

  • Uniform Camera Motion Compensation (UCMC): Unlike standard CMC that calculates transformations for every frame pair, UCMCTrack applies a consistent set of compensation parameters across an entire video sequence, drastically reducing computational overhead.
  • Mapped Mahalanobis Distance (MMD): This is the system's primary innovation. Instead of binary IoU, MMD calculates the distance between a predicted track and a new detection by projecting probability distributions onto the ground plane. It accounts for the uncertainty and measurement noise inherent in homography projections.
  • Process Noise Compensation (PNC): The authors modify the Kalman Filter’s process noise matrix to specifically account for camera-induced jitter and rotation, allowing the motion model to remain flexible during sudden movements.
  • Ground Plane Modeling: By using a homography matrix to project the bottom-center of bounding boxes to ground coordinates $(x, y)$, the tracker leverages the fact that real-world human motion is more linear and predictable on the floor than on a 2D screen.

Datasets and Evaluation

The researchers evaluated UCMCTrack on four major benchmarks:

  • MOT17 & MOT20: Standard benchmarks for pedestrian tracking in crowded environments.
  • DanceTrack: A challenging dataset featuring targets with similar appearances and highly irregular, non-linear motion.
  • KITTI: A classic autonomous driving dataset used to test vehicle and pedestrian tracking from a moving platform.

Key Insights

  • Beyond IoU: The paper demonstrates that IoU is a fragile metric in dynamic scenes. MMD provides a continuous distance metric that allows for successful data association even when there is zero spatial overlap between bounding boxes.
  • Motion-Only Efficacy: Remarkably, UCMCTrack achieves state-of-the-art results using motion cues alone, without relying on heavy deep-learning-based appearance features (ReID). This proves that sophisticated motion modeling can often outperform complex visual descriptors.
  • The Power of the Ground Plane: Tracking on the ground plane naturally handles perspective distortion. An object moving at a constant speed in the real world has a constant velocity on the ground plane, whereas its velocity on the image plane changes based on its distance from the camera.

Key Results

UCMCTrack demonstrates exceptional performance and efficiency. On the MOT17 test set, UCMCTrack+ achieved 65.8 HOTA and 81.1 IDF1, surpassing leading trackers like Bot-SORT and StrongSORT.

Performance comparison on the MOT17 test set, highlighting UCMCTrack's superior HOTA and IDF1 scores.

Furthermore, it showed a significant lead on DanceTrack (63.6 HOTA), a dataset specifically designed to defeat trackers that rely too heavily on appearance. When provided with detections, the tracking association itself runs at over 1000 FPS on a single CPU, making it ideal for real-world deployment where compute resources are limited.

Limitations and Future Work

The primary limitation of UCMCTrack is its single ground plane assumption. The current model assumes all targets reside on a flat, singular plane, which may cause issues in environments with significant elevation changes, such as multi-level parking garages. Future work could involve integrating UCMCTrack with lightweight ReID modules or exploring multi-plane projections to handle more complex 3D environments.

Summary updated

BibTeX
@inproceedings{yi2024ucmctrack,
  author = {Kefu Yi and Kai Luo and Xiaolei Luo and Jiangui Huang and Hao Wu and Rongdong Hu and Wei Hao},
  title = {{UCMCTrack: Multi-Object Tracking with Uniform Camera Motion Compensation}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2024},
  doi = {10.1609/aaai.v38i7.28493},
}
Figure