TL;DR
This paper tackles the computational bottleneck in real-time multi-object tracking (MOT) by proposing a novel system that uses an ensemble of object detectors, each running at a reduced frame rate. This approach achieves significantly faster processing speeds than existing methods on the MOT16 benchmark, while maintaining competitive accuracy, making real-time MOT applications more feasible.
What Problem Does This Paper Address?
Multiple-Object Tracking (MOT) is essential for applications like video surveillance and retail analytics. A common MOT approach is "tracking-by-detection," which involves two steps: detecting objects and then associating these detections to form trajectories. The primary challenge in this paradigm is the computational cost of object detection, which often runs on every frame, making real-time tracking difficult. This paper aims to overcome this by reducing the computational load of the detection stage without sacrificing too much accuracy.
Methods
The core innovation lies in an object detection ensemble that runs every f frames, rather than every frame. This ensemble combines predictions from multiple object detectors. To reconcile detections from different detectors and handle overlapping bounding boxes, they adapt the Soft-NMS (Non-Maximum Suppression) algorithm. The tracking itself uses a Kalman filter-based approach, similar to existing methods, but benefits from the reduced detection load. The system models object states and transitions, and uses a cost-based assignment problem to match detections to existing tracks. A key aspect is the use of a statistical distance measure that accounts for uncertainty, improving robustness.
Datasets
The paper evaluates its system on the MOT16 benchmark, a standard dataset for multi-object tracking. No new datasets were introduced.
Key Insights and Ideas
- Reduced Detection Frequency is Key: Running object detectors less frequently (every f frames) is a viable strategy to achieve real-time performance without a drastic drop in accuracy.
- Ensemble Power: Combining multiple detectors, even if run less often, can improve overall detection robustness and reduce false negatives compared to a single detector.
- Soft-NMS Adaptation: Adapting Soft-NMS for combining ensemble detections is an effective way to merge and score overlapping bounding boxes from different models.
- Trade-off Management: The paper explicitly demonstrates the trade-off between frame skipping (f) and accuracy, showing that a moderate skip rate (f=5) offers a good balance.
Key Results
The proposed system achieved the highest speed among online entries on the MOT16 challenge, running at 249.6 Hz (when considering only the tracker, not the detector ensemble). When the ensemble detector was included, the system still achieved real-time performance. For instance, with f=5, the system achieved a competitive MOTA score while significantly outperforming other methods in speed. The ensemble detector itself showed an average accuracy improvement of 7.8% (MOTA) over the best individual detector when f=1, and 5.5% when f=5.
Limitations and Future Work
The authors acknowledge that skipping frames (f > 1) leads to a degradation in accuracy. They also note that their system, while fast, does not incorporate person re-identification, which could further improve tracking robustness, especially in cases of long-term occlusion. Future work directions include:
- Improving the persistence of tracked objects.
- Reducing reliance on perfect object detectors.
- Incorporating more object detectors into the ensemble.
- Adding person re-identification capabilities.
Summary updated