Abstract
Multi-object tracking (MOT) has seen significant progress, yet most state-of-the-art trackers are optimized for predictable pedestrian movement. Generic objects, such as animals or athletes, often exhibit "jumpy" and non-linear motion that leads to tracking failures in standard Intersection over Union (IoU) based systems. We present FRoG-MOT, a framework that represents motion through multiple independent states—velocity, acceleration, and shape—and adaptively discards outlier states during association. This approach achieves state-of-the-art robustness and trajectory continuity on generic benchmarks while maintaining real-time processing speeds of 28.2 FPS.
Introduction
Modern MOT benchmarks like MOT17 are dominated by pedestrians with linear, predictable trajectories. In these cases, simple IoU-based association is highly effective. However, generic objects in the wild (e.g., flocks of birds, schools of fish) do not follow these patterns; they change direction instantly and alter their body shape rapidly. These sudden motion changes cause IoU-based trackers to lose targets, resulting in fragmented trajectories and frequent identity swaps. FRoG-MOT addresses this gap by providing a tracker robust to sudden motion changes without the computational overhead of complex transformer-based architectures.
Methodology
FRoG-MOT follows the "tracking-by-detection" paradigm but introduces a sophisticated motion-handling layer to improve association robustness.
Multiple Motion States
The system tracks three distinct attributes for every object:
- Shape: Changes in aspect ratio and bounding box size.
- Velocity: The first-order derivative of position.
- Acceleration: The second-order derivative of position.
Robust Association and Outlier Rejection
Standard trackers often combine motion cues multiplicatively. If a single cue (like acceleration) spikes, the entire association probability can drop, breaking the track. FRoG-MOT employs an L-estimator-inspired approach that calculates similarity for all three states but ignores the outlier (the state with the lowest similarity). By relying on the "best two" cues, the tracker maintains lock even if one attribute changes abruptly.
Adaptive Motion State Modification
When a motion state is flagged as an outlier, the system uses a parametric t-test to determine if the object has entered a new movement "mode." If the change is statistically significant, the internal motion model is updated to reflect this new behavior, allowing the tracker to adapt to changing dynamics in real-time.
Key Insights: Generic vs. Human Motion
A core contribution of this work is the quantitative analysis of motion variance across different datasets. The authors demonstrate that generic objects exhibit significantly higher variance in velocity and acceleration—up to 10 times higher—than pedestrians in the MOT17 dataset.
This variance explains why pedestrian-tuned trackers fail on generic tasks. Furthermore, the redundancy provided by tracking three motion attributes allows the system to remain resilient. For instance, if a bird flaps its wings (changing its shape) but maintains its flight path, the tracker prioritizes velocity and acceleration to maintain the identity.
Experimental Results
The researchers evaluated FRoG-MOT across several benchmarks, including MOT17, GMOT-40, and a new standardized benchmark, GMOT-Split101.
- Continuity: On GMOT-Split101, FRoG-MOT achieved an IDF1 score of 80.8, outperforming ByteTrack (78.8) and Trackformer (78.5).
- Efficiency: The system runs at 28.2 FPS, making it suitable for real-time applications.
- Low FPS Resilience: The advantage of FRoG-MOT over IoU-only trackers increases as the video frame rate decreases. In low-FPS scenarios where IoU overlap is minimal, robust motion modeling becomes the primary driver of tracking success.
Conclusion
FRoG-MOT provides a fast and robust solution for tracking generic objects with unpredictable motion. By leveraging redundant motion states and adaptive outlier rejection, it bridges the gap between simple IoU trackers and heavy transformer-based models. While it remains dependent on the quality of the external detector, its ability to handle complex motion patterns makes it a significant step forward for generic multi-object tracking.
Summary updated