FRoG-MOT: Fast and Robust Generic Multiple-Object Tracking by IoU and Motion-State Associations

Takuya Ogawa, Takashi Shibata, Toshinori Hosoi

Abstract

Multi-object tracking (MOT) has seen significant progress, yet most state-of-the-art trackers are optimized for predictable pedestrian movement. Generic objects, such as animals or athletes, often exhibit "jumpy" and non-linear motion that leads to tracking failures in standard Intersection over Union (IoU) based systems. We present FRoG-MOT, a framework that represents motion through multiple independent states—velocity, acceleration, and shape—and adaptively discards outlier states during association. This approach achieves state-of-the-art robustness and trajectory continuity on generic benchmarks while maintaining real-time processing speeds of 28.2 FPS.

Introduction

Modern MOT benchmarks like MOT17 are dominated by pedestrians with linear, predictable trajectories. In these cases, simple IoU-based association is highly effective. However, generic objects in the wild (e.g., flocks of birds, schools of fish) do not follow these patterns; they change direction instantly and alter their body shape rapidly. These sudden motion changes cause IoU-based trackers to lose targets, resulting in fragmented trajectories and frequent identity swaps. FRoG-MOT addresses this gap by providing a tracker robust to sudden motion changes without the computational overhead of complex transformer-based architectures.

Methodology

FRoG-MOT follows the "tracking-by-detection" paradigm but introduces a sophisticated motion-handling layer to improve association robustness.

Multiple Motion States

The system tracks three distinct attributes for every object:

  1. Shape: Changes in aspect ratio and bounding box size.
  2. Velocity: The first-order derivative of position.
  3. Acceleration: The second-order derivative of position.

Robust Association and Outlier Rejection

Standard trackers often combine motion cues multiplicatively. If a single cue (like acceleration) spikes, the entire association probability can drop, breaking the track. FRoG-MOT employs an L-estimator-inspired approach that calculates similarity for all three states but ignores the outlier (the state with the lowest similarity). By relying on the "best two" cues, the tracker maintains lock even if one attribute changes abruptly.

The FRoG-MOT framework integrates IoU-based and motion-state associations with adaptive state modification.

Adaptive Motion State Modification

When a motion state is flagged as an outlier, the system uses a parametric t-test to determine if the object has entered a new movement "mode." If the change is statistically significant, the internal motion model is updated to reflect this new behavior, allowing the tracker to adapt to changing dynamics in real-time.

Key Insights: Generic vs. Human Motion

A core contribution of this work is the quantitative analysis of motion variance across different datasets. The authors demonstrate that generic objects exhibit significantly higher variance in velocity and acceleration—up to 10 times higher—than pedestrians in the MOT17 dataset.

Comparison of motion statistics between MOT17 (pedestrians) and GMOT-40 (generic objects), highlighting the higher variance in generic motion.

This variance explains why pedestrian-tuned trackers fail on generic tasks. Furthermore, the redundancy provided by tracking three motion attributes allows the system to remain resilient. For instance, if a bird flaps its wings (changing its shape) but maintains its flight path, the tracker prioritizes velocity and acceleration to maintain the identity.

Experimental Results

The researchers evaluated FRoG-MOT across several benchmarks, including MOT17, GMOT-40, and a new standardized benchmark, GMOT-Split101.

  • Continuity: On GMOT-Split101, FRoG-MOT achieved an IDF1 score of 80.8, outperforming ByteTrack (78.8) and Trackformer (78.5).
  • Efficiency: The system runs at 28.2 FPS, making it suitable for real-time applications.
  • Low FPS Resilience: The advantage of FRoG-MOT over IoU-only trackers increases as the video frame rate decreases. In low-FPS scenarios where IoU overlap is minimal, robust motion modeling becomes the primary driver of tracking success.

Conclusion

FRoG-MOT provides a fast and robust solution for tracking generic objects with unpredictable motion. By leveraging redundant motion states and adaptive outlier rejection, it bridges the gap between simple IoU trackers and heavy transformer-based models. While it remains dependent on the quality of the external detector, its ability to handle complex motion patterns makes it a significant step forward for generic multi-object tracking.

Summary updated

Abstract

This paper proposes a generic multi-object tracking (MOT) algorithm that is robust to unexpected motion changes for generic objects. Deep learning has dramatically been improving MOT performances. Nevertheless, state-of-the-art tracking algorithms are still sensitive to unexpected motion changes and the generic object target beyond person tracking. This is because standard MOT benchmark datasets such as MOT17 mainly consist of persons in a crowd, often lacking unexpected shape and motion changes; thus, these issues have yet to be focused on. We propose a simple-yet-effective MOT framework that can dynamically improve tracking continuity by associating each target based on adaptively modified motion states. The keys are 1) to represent the target motions using multiple motion states that have weak correlations with each other and 2) to modify those states that have the lowest similarity to past states as outliers. Our approach can improve trajectory continuity and robustness to unexpected motion changes for generic objects. Comprehensive experiments have confirmed that our framework is comparable to existing state-of-the-art methods on a standard dataset and outperforms those algorithms on the GMOT dataset with an overall 2% improvement in IDF1, a measure of tracking continuity.
BibTeX
@inproceedings{ogawa2024frog,
  author = {Takuya Ogawa and Takashi Shibata and Toshinori Hosoi},
  title = {{FRoG-MOT: Fast and Robust Generic Multiple-Object Tracking by IoU and Motion-State Associations}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2024},
  doi = {10.1109/wacv57701.2024.00643},
}
Figure