TL;DR
This paper investigates whether pre-processing underwater images with enhancement algorithms actually improves object detection performance. Surprisingly, the study finds that enhancement often hinders object detection, particularly for challenging underwater scenes, and that current image quality metrics don't reliably predict detection performance.
What Problem Does This Paper Address?
Underwater object detection is vital for marine engineering and robotics, but raw underwater images are notoriously difficult to work with due to poor visibility, color distortion, and scattering. A common intuition is that enhancing these images before feeding them to object detectors should improve performance. However, there's a lack of rigorous, comprehensive studies to confirm this. This paper aims to answer two critical questions: "Does underwater image enhancement really improve underwater object detection?" and "How does it contribute (or detract) from it?" By systematically evaluating a wide range of enhancement techniques, the authors seek to clarify this relationship and guide future research.
Methods
The authors conducted an extensive empirical study. They selected 18 state-of-the-art underwater image enhancement algorithms, spanning traditional methods, CNN-based approaches, and GANs. These algorithms were used to process existing underwater object detection datasets. Then, 7 popular deep learning-based object detectors (both two-stage and one-stage, anchor-based and anchor-free) were retrained on both the original (raw) and the enhanced datasets. This resulted in a total of 133 trained models. The performance of these models was then analyzed using various quantitative metrics (like AP, TIDE) and qualitative assessments, including Class Activation Maps (CAMs) to visualize feature importance. Complementary experiments were also performed using stronger backbones and longer training schedules to see if they could mitigate any negative effects of enhancement.
Datasets
The study primarily used two datasets:
- URPC2020: A dataset containing 5,543 underwater images with four categories: holothurian, echinus, scallop, and starfish. It was split into training and testing sets.
- RUOD (Real-world Underwater Object Detection): A larger dataset with 14,000 images and over 74,000 bounding boxes across ten categories, including fish, echinus, corals, starfish, holothurian, scallop, diver, cuttlefish, turtle, and jellyfish.
Key Insights and Ideas
- Enhancement is often detrimental: The most striking finding is that, contrary to intuition, most underwater image enhancement techniques suppress object detection performance compared to using raw images.
- Hard cases are most affected: The negative impact of enhancement is most pronounced on challenging underwater scenes and "harder" samples, where it seems to introduce more noise and interference, confusing the detectors.
- Visual quality ≠ Detection performance: Current image quality assessment metrics (like AG, EI, UIQM) don't reliably correlate with object detection performance. An image might look better but lead to worse detection.
- Color cast isn't the main culprit: While color cast is a common underwater image problem, it's not the primary reason enhancement hurts detection. Other factors like introduced noise, edge blur, and texture corruption are more impactful.
- Over-processing is harmful: Enhancing image properties like saturation and color richness too much can degrade detector performance.
- Stronger detectors don't fix it: Using more powerful backbones or longer training schedules for detectors doesn't overcome the negative impact of image enhancement.
Key Results
- Raw data wins: In almost all tested scenarios, detectors trained on raw underwater images achieved the highest Average Precision (AP) scores.
- Performance degradation: For example, on the URPC2020 dataset, detectors retrained on enhanced images often showed a decrease in AP compared to those trained on raw images. Some enhancement methods led to AP drops of several percentage points.
- Error analysis: TIDE analysis revealed that enhancement significantly increases background errors (detecting background as objects) and missed detections, which are the most critical factors affecting AP.
- Qualitative examples: Visual results (Fig. 4) show that while raw images might be visually poor, detectors trained on them often correctly identify objects, whereas enhanced images lead to missed detections or false positives.
Limitations and Future Work
- Limitations: The study acknowledges that the URPC2020 dataset has a relatively simple background, and the RUOD dataset, while larger, might still have limitations. The analysis focuses on pre-processing; end-to-end joint optimization of enhancement and detection is a separate area.
- Future Work: The authors suggest several directions:
- Developing enhancement algorithms with better generalization, avoiding introduced noise and color casts, and specifically targeting background interference.
- Creating new evaluation metrics for image enhancement that better align with human perception and the requirements of downstream tasks like object detection.
- Designing "task-oriented" enhancement methods and exploring joint optimization strategies for image enhancement and object detection, especially for low-quality image scenarios.
Summary updated