Abstract: Multi-object tracking (MOT) enables autonomous vehicles to continuously perceive dynamic objects, supplying essential temporal cues for prediction, behavior understanding, and safe planning. However, conventional tracking-by-detection methods typically rely on static coordinate transformations based on ego-vehicle poses, disregarding ego-vehicle speed-induced variations in observation noise and reference frame changes, which degrades tracking stability and accuracy in dynamic, high-speed scenarios. In this paper, we investigate the critical role of ego-vehicle speed in MOT and propose a Speed-Guided Learnable Kalman Filter (SG-LKF) that dynamically adapts uncertainty modeling to ego-vehicle speed, significantly improving stability and accuracy in highly dynamic scenarios. Central to SG-LKF is MotionScaleNet (MSNet), a decoupled token-mixing and channel-mixing MLP that adaptively predicts key parameters of SG-LKF. To enhance inter-frame association and trajectory continuity, we introduce a self-supervised trajectory consistency loss jointly optimized with semantic and positional constraints. Extensive experiments show that SG-LKF ranks first among all vision-based methods on KITTI 2D MOT with 79.59% HOTA, delivers strong results on KITTI 3D MOT with 82.03% HOTA, and outperforms SimpleTrack by 2.2% AMOTA on nuScenes 3D MOT.
Introduction
In the autonomous driving domain, Multi-Object Tracking (MOT) typically follows the "Tracking-by-Detection" (TBD) paradigm. This involves identifying objects in individual frames and using a motion model—most commonly a Kalman Filter (KF)—to predict their future positions for inter-frame association. While effective in controlled environments, standard Kalman Filters often struggle in the highly dynamic conditions of real-world driving.
The Speed-Dependency Problem
Traditional Kalman Filters rely on fixed parameters for process noise ($Q$) and observation noise ($R$). These constants assume a stable relationship between frames. However, as the ego-vehicle accelerates, several factors degrade tracking performance:
- Reference Frame Shifts: Rapid coordinate system changes make linear motion assumptions less accurate.
- Greater Displacement: Objects move further between frames, complicating the matching of detections to existing tracks.
- Observation Noise: High speeds introduce motion blur and rapid viewpoint changes, reducing detection reliability.
Empirical analysis shows that in top-tier trackers like BoT-SORT, the "Identity Switch" rate spikes and the "Intersection over Union" (IoU) of predicted boxes drops significantly as the ego-vehicle speeds up. This paper addresses these issues by treating ego-vehicle speed as a critical prior for dynamic uncertainty modeling.
Methodology: Speed-Guided Learnable Kalman Filter
The core contribution of this work is the Speed-Guided Learnable Kalman Filter (SG-LKF). Unlike "black box" neural trackers that replace the Kalman Filter entirely, SG-LKF preserves the filter's robust mathematical structure while making its parameters learnable and adaptive to driving conditions.
MotionScaleNet (MSNet)
To dynamically adjust the Kalman Filter, the authors introduced MotionScaleNet (MSNet), a lightweight (78.2K parameters) decoupled token-mixing and channel-mixing MLP. MSNet takes three key inputs—ego-vehicle speed, object width, and object height—to predict optimal $P, Q,$ and $R$ matrices on the fly. By including object scale, the model learns that different entities (e.g., a large truck vs. a small pedestrian) require distinct uncertainty models at high speeds.
Trajectory Consistency Loss (TCL)
To ensure smooth and continuous tracking, the system is optimized using a self-supervised Trajectory Consistency Loss (TCL). This loss function enforces two types of stability:
- Positional Consistency: Ensuring the object's path remains physically plausible over time.
- Semantic Consistency: Leveraging DINOv2 features to ensure the visual identity of an object remains stable, even if its geometric bounding box becomes jittery due to noise.
A "Mamba-inspired" temporal accumulation mechanism is used to look at long-term trajectory history, helping the tracker maintain identities through brief occlusions or sensor noise.
Experimental Results
The SG-LKF method was evaluated on major autonomous perception benchmarks, demonstrating superior performance and robustness:
- KITTI 2D MOT: Ranked 1st among all vision-based methods with a 79.59% HOTA (Higher Order Tracking Accuracy).
- KITTI 3D MOT: Achieved 82.03% HOTA, outperforming complex fusion-based models.
- nuScenes: Improved the AMOTA (Average Multi-Object Tracking Accuracy) of the baseline by 2.2%, while significantly reducing tracking precision errors (AMOTP).
Robustness tests further confirmed that the system remains stable even when speed sensor data is corrupted with Gaussian noise, proving its reliability for real-world deployment.
Conclusion
SG-LKF demonstrates that ego-vehicle speed is a vital but often overlooked signal in multi-object tracking. By integrating this signal into a learnable Kalman Filter framework via MSNet, the proposed method achieves state-of-the-art results and maintains high stability across a wide range of driving speeds. Future work will explore more advanced architectures to handle dense object interactions and extreme swerving scenarios.
Summary updated