Abstract
Autonomous navigation for unmanned surface vessels (USVs) requires robust Multi-Object Tracking (MOT) to ensure safety in dynamic maritime environments. However, traditional MOT methods often struggle with platform instability, visual degradation, and computational constraints. This paper presents DMSORT (Dual-branch Maritime SORT), a parallel tracking architecture utilizing a Detection-driven Dual-Task Association (DDTA) framework. By integrating a Reversible Columnar Detection Network (RCDN) and a lightweight Transformer-based appearance extractor (Li-TAE), DMSORT achieves state-of-the-art accuracy (76.69% HOTA on the Singapore Maritime Dataset) while maintaining real-time performance at 30.6 FPS on edge-grade hardware.
Introduction: The Challenges of Maritime Tracking
Multi-Object Tracking (MOT) serves as the primary sensory system for autonomous platforms, enabling the identification and tracking of multiple targets across video frames. While MOT has matured in urban environments for self-driving cars, the maritime domain presents a "perfect storm" of unique obstacles:
- Platform Instability: Wave-induced camera jitter disrupts standard motion models.
- Visual Degradation: Environmental factors like fog, sea glare, and low resolution obscure targets.
- Scale Variance: Scenes often contain objects ranging from massive cargo ships to small, bobbing buoys.
- Onboard Compute Constraints: USVs require energy-efficient models for real-time processing.
Existing approaches typically fall into Tracking-by-Detection (TBD) or Joint Detection and Tracking (JDT). TBD is accurate but suffers from sequential latency, while JDT is fast but prone to "gradient interference" during training, where detection and tracking tasks conflict.
Methods: The DMSORT Architecture
DMSORT addresses these limitations through the DDTA framework, which allows detection and tracking tasks to proceed in parallel rather than sequentially.
Key Technical Innovations
- Reversible Columnar Detection Network (RCDN): RCDN serves as the backbone, employing a reversible structure that recomputes features on-the-fly during training. This significantly reduces memory overhead while preserving fine-grained spatial details necessary for detecting small maritime targets.
- Li-TAE (Lightweight Transformer-based Appearance Extractor): Utilizing a "shrink-attention" Vision Transformer, Li-TAE captures global context to maintain target identity even during partial occlusions or visual degradation.
- Affine Compensation: To mitigate camera motion from waves, the system estimates camera ego-motion and applies affine transformations to the Kalman filter, stabilizing target trajectories.
- Clustering Optimized Feature Fusion (COFF): COFF balances motion and appearance cues using a nonlinear scaling mechanism. By amplifying appearance similarity scores—which are typically orders of magnitude smaller than motion scores—it prevents the tracker from over-relying on simple movement, thereby reducing identity switches.
Experimental Evaluation
The researchers primarily used the Singapore Maritime Dataset (SMD), a benchmark featuring high-definition video from shore-based and USV-mounted cameras in various weather conditions. To demonstrate robustness, DMSORT was also evaluated on the MOT17 pedestrian dataset.
Key Results
DMSORT sets a new standard for maritime tracking:
- State-of-the-Art Performance: On the SMD, the enhanced version (DMSORT) achieved a HOTA (Higher Order Tracking Accuracy) of 76.69%*, outperforming established trackers like ByteTrack and OCSORT.
- Real-Time Speed: The system achieves 30.6 FPS on a laptop-grade GPU (RTX 4060), making it viable for real-time deployment on unmanned vessels.
- Generalization: Despite its maritime focus, DMSORT achieved the highest MOTA (78.60) on the MOT17 dataset, proving the architecture's versatility across different domains.
Conclusion
DMSORT provides a high-performance, memory-efficient solution for maritime MOT. By decoupling detection and tracking and employing a reversible backbone, it overcomes the traditional trade-offs between speed and accuracy. Future work will explore multimodal fusion—integrating radar and LiDAR data—to enhance reliability in extreme weather and zero-visibility conditions.
Summary updated