TL;DR
This paper introduces U-DECN, a novel end-to-end underwater object detection system built on a ConvNet architecture. It tackles the challenges of underwater color cast noise and the need for efficient deployment on resource-constrained platforms, achieving state-of-the-art performance with significantly improved speed.
What Problem Does This Paper Address?
Underwater object detection is crucial for ocean exploration but faces significant hurdles. The unique optical properties of water, such as light absorption and scattering, introduce color cast noise that degrades image quality and severely impacts detection accuracy. Furthermore, deploying these detectors on Unmanned Underwater Vehicles (UUVs) is challenging due to their limited processing power and the requirement for real-time performance. Existing methods, including transformer-based and traditional CNN-based detectors, often struggle with either accuracy in noisy conditions, computational efficiency, or both, and many still rely on computationally expensive post-processing steps like Non-Maximum Suppression (NMS).
Methods
The U-DECN architecture builds upon the DECO framework, a ConvNet-based end-to-end object detector, and integrates advancements from transformer-based detectors like DINO. Key innovations include:
- Hybrid Encoder: To address the lack of multi-scale feature information in DECO's single-level feature processing, U-DECN incorporates a hybrid encoder that fuses features from multiple scales, inspired by RT-DETR.
- Two-Stage Bounding Box Refinement: This technique, borrowed from DINO, improves model performance by refining bounding box predictions twice per decoder layer.
- Deformable Convolution in SIM: To mitigate the "query feature misalignment issue" that arises when dynamically initialized positional queries interact with ConvNet features, the authors replace the standard convolution in the Self-Interaction Module (SIM) with deformable convolution (DCNv3). This allows queries to adapt their positions for better feature alignment.
- Underwater Color DeNoising Query (ColorDN): This novel query mechanism encodes prior information about underwater color cast noise (derived from the mean RGB channels of the image) to improve the model's generalization to biased object features caused by different color casts.
- Separate Contrastive DeNoising Forward: To enable contrastive denoising training within the ConvNet decoder, which has fixed tensor shape requirements, this method processes object queries and contrastive denoising queries separately, allowing for effective denoising training and faster convergence.
Datasets
The paper evaluates U-DECN on two underwater object detection datasets:
- DUO [17]: Consists of training and test sets with 4 categories of underwater organisms.
- RUOD [8]: Features training and test sets with 10 categories of underwater objects across various scenarios.
Key Insights and Ideas
- ConvNets can be competitive for end-to-end detection: By carefully adapting techniques from transformer architectures, the authors demonstrate that ConvNet-based models can achieve state-of-the-art performance in end-to-end object detection, offering a more efficient alternative.
- Deformable convolution is key for query alignment in ConvNets: The "query feature misalignment issue" is a critical bottleneck for ConvNet-based end-to-end detectors when using advanced query initialization techniques. Deformable convolution effectively addresses this by allowing queries to adapt their spatial positions.
- Explicitly modeling color cast noise improves generalization: Instead of just enhancing images, encoding color cast information as a dedicated query (ColorDN) directly improves the model's ability to handle biased feature information caused by different color casts.
- Separate training paths for queries are necessary for ConvNet decoders: The fixed structure of ConvNet decoders requires a tailored approach to contrastive denoising training, as demonstrated by the "Separate Contrastive DeNoising Forward" method.
Key Results
- State-of-the-art performance: U-DECN achieves the best AP on the DUO dataset (64.0 AP) and competitive results on RUOD (58.1 AP).
- Significant speedup: On an NVIDIA AGX Orin with TensorRT FP16, U-DECN achieves 21 FPS, which is 5 times faster than Deformable DETR and DINO (both 4 FPS). This makes it highly suitable for real-time deployment on UUVs.
- Improved training convergence: The proposed methods, particularly Separate Contrastive DeNoising Forward, lead to faster training convergence compared to baseline DECO.
- Robustness to color casts: U-DECN shows improved generalization performance across different underwater color cast conditions.
Limitations and Future Work
The authors acknowledge a few limitations:
- Query count impact: Increasing the number of queries beyond a certain point makes model learning difficult and reduces performance, a limitation more pronounced in ConvNet architectures compared to transformers.
- Limited to color cast noise: U-DECN is designed to handle color cast noise but does not address other underwater environmental challenges like suspended solids, uneven lighting, or refraction.
Future work could explore addressing the query count limitation and extending the model's robustness to a wider range of underwater noise conditions.
Summary updated