Abstract
One-shot Multi-Object Tracking (MOT) models often suffer from performance degradation due to the inherent competition between object detection and re-identification (ReID) tasks. This paper introduces CSTrack, a novel tracker designed to mitigate this "tug-of-war" through a Reciprocal Network (REN) for task decoupling and a Scale-Aware Attention Network (SAAN) for scale-invariant identity embeddings. CSTrack achieves state-of-the-art results on standard benchmarks while maintaining real-time inference speeds, effectively bridging the gap between one-shot efficiency and two-stage accuracy.
Introduction: The One-Shot Dilemma
In Multi-Object Tracking, researchers typically choose between two paradigms: two-stage and one-shot. Two-stage methods detect objects first and then run a separate model for identity feature extraction (ReID). While accurate, they are computationally expensive. One-shot methods integrate both tasks into a single network for speed, but they historically struggle with accuracy.
The authors identify a fundamental conflict in one-shot architectures: Detection requires the network to recognize all pedestrians as similar (high intra-class similarity), whereas ReID requires the network to distinguish between individuals (high intra-class variance). When these tasks share a feature space, they compete, leading to "ambiguous learning."
Methods: Decoupling and Scale Awareness
CSTrack addresses these issues through two primary architectural innovations built upon a YOLO-v5-style backbone:
- Reciprocal Network (REN): Instead of simply splitting the network head, REN utilizes "self-relation" and "cross-relation" layers. Self-relation allows each branch to specialize in task-specific features, while cross-relation enables the exchange of helpful common information without interference. This effectively decouples the competing objectives.
- Scale-Aware Attention Network (SAAN): Standard feature pyramids (FPNs) often assign different-sized objects to different layers, causing "semantic misalignment" for ReID. A person walking toward the camera might look different to the model simply because they moved to a higher-resolution layer. SAAN aggregates features from multiple resolutions using spatial and channel attention, ensuring that identity embeddings remain consistent regardless of the object's scale in the frame.
Key Results
CSTrack was evaluated on the MOT16, MOT17, and MOT20 benchmarks, achieving state-of-the-art performance:
- Accuracy: On MOT17, it achieved a 74.9 MOTA (Multiple Object Tracking Accuracy) and 72.3 IDF1 (Identification F1 score), outperforming previous leading models like FairMOT and JDE.
- Efficiency: The standard model runs at 16.4 FPS, while a lightweight version (CSTrack-S) reaches 34.6 FPS, making it suitable for real-time deployment.
- Discriminative Power: In upper-bound tests using ground-truth detections, CSTrack’s ReID features proved significantly more discriminative than prior one-shot methods, demonstrating a cleaner diagonal on cosine similarity matrices.
Conclusion
By rethinking the relationship between detection and ReID, CSTrack proves that one-shot trackers can achieve high accuracy without sacrificing speed. The introduction of REN and SAAN provides the necessary architectural "breathing room" for task specialization and scale invariance. Future work will focus on incorporating temporal information to improve tracking robustness against image blur and long-term occlusions.
Summary updated