Introduction
Multi-person pose tracking is a critical task in computer vision, yet achieving high accuracy in real-time remains challenging. Traditional "top-down" methods are accurate but slow due to their multi-stage nature. Conversely, unified frameworks are faster but often struggle with scale variation. FastPose bridges this gap by integrating human detection, pose estimation, and person re-identification (Re-ID) into a single, efficient multi-task network.
Methods: Multi-tasking and Scale Normalization
The primary hurdles in unified pose tracking are scale variation—where people appear at different sizes—and occlusions. FastPose introduces three key innovations to solve these:
- Multi-task Network (MTN): A single backbone (e.g., ResNet or MobileNet) supports three specialized heads for detection, 2D pose estimation, and Re-ID feature extraction. This shared architecture significantly reduces the marginal cost of adding tracking and pose estimation to a detector.
- Scale-normalized Image and Feature Pyramid (SIFP): This strategy ensures the network learns to process humans at consistent, optimal resolutions by ignoring extreme scales during specific training phases. During inference, it handles different sizes without the overhead of traditional image pyramids.
- Occlusion-aware Re-ID Strategy: By using pose estimation outputs to assess the visibility of keypoints, the system can determine if a Re-ID feature is reliable. If too many joints are hidden, the system adjusts its tracking logic to avoid errors during occlusions.
Experimental Results
FastPose was evaluated on standard benchmarks like COCO, MPII, and PoseTrack. The results demonstrate a superior balance between speed and accuracy compared to existing state-of-the-art methods.
- Real-time Performance: With a ResNet-18 backbone, FastPose achieves 29.4 FPS, comfortably meeting real-time requirements. Even with a heavier ResNet-50 backbone, it maintains 12.2 FPS.
- High Accuracy: The ResNet-101 variant reaches 70.3 mAP on the PoseTrack validation set, outperforming many complex, non-real-time models.
- Robust Tracking: The occlusion-aware strategy reduces ID switches (where the tracker loses track of individuals) by 37%, proving the effectiveness of pose-guided tracking.
Conclusion
FastPose demonstrates that a unified multi-task network can match the accuracy of complex two-stage systems while maintaining real-time speeds. By focusing on scale normalization and leveraging pose information for tracking, it provides a robust framework for dynamic video analysis. Future work may extend this approach to 3D pose estimation and more complex crowded environments.
Summary updated