TL;DR
This paper introduces UIERL, a novel network for underwater image enhancement that tackles two key challenges: the varying quality within a single image due to depth differences and the underutilization of information from related images. By jointly learning from internal image characteristics and external related images, UIERL achieves state-of-the-art performance in restoring clarity and color to underwater scenes.
What Problem Does This Paper Address?
Underwater environments present unique challenges for image capture. Light absorption and scattering cause significant degradation, leading to color casts, low contrast, and blurred details. A critical issue is that these distortions aren't uniform across an image; regions at different depths experience varying levels of degradation. Existing methods often treat the entire image uniformly, failing to address these internal quality variations. Furthermore, underwater imaging systems (like AUVs and ROVs) often capture multiple images of similar scenes. These correlated images contain rich, complementary information that could significantly aid in enhancing a single target image, but current single-image enhancement techniques largely ignore this external context.
Methods
UIERL employs a two-stage approach:
- Internal Representation Learning Stage: This stage focuses on addressing the region-wise quality differences within a single image. It introduces a Depth-based Region Feature Guidance (DRFG) network. DRFG first estimates the scene depth and then segments the image into regions with similar degradation patterns. A region-wise space encoder then learns features independently for each region, allowing for differentiated enhancement. This provides a more nuanced guidance for global feature learning.
- External Representation Learning Stage: This stage leverages information from related underwater images. An External Information Extraction Network (EIEN) is proposed to mine features from these related images. Crucially, UIERL then facilitates interaction between the internal features of the target image and the external features from related images through two novel modules:
- External-Assist-Internal (EAI) module: Uses external features to update internal features.
- Internal-Assist-External (IAE) module: Uses internal features to update external features. This bidirectional interaction allows for a comprehensive exploitation of both internal and external information.
Datasets
The paper uses several datasets for training and evaluation:
- UISD Dataset: A newly developed large-scale dataset containing multiple images per scene, designed to better reflect real-world underwater applications. It consists of 313 scenes (1571 images) for training and 60 scenes (271 images) for testing.
- SQUID: A dataset of 57 underwater stereo image pairs.
- sub-RUIE, sub-UIEB, sub-EUVP: Subsets derived from commonly used underwater image enhancement datasets, chosen for their relevance to practical applications.
- UISD-1200: The remaining 1200 scenes from the proposed UISD dataset, used for validation.
Key Insights and Ideas
- Depth as a Degradation Indicator: The paper highlights that scene depth is a primary driver of varying image quality in underwater scenes, offering a direct way to segment and address these internal differences.
- Leveraging Correlation is Key: The inherent correlation between images captured in similar underwater scenes is a rich, untapped resource for improving single-image enhancement.
- Synergistic Internal-External Learning: Simply processing internal or external information isn't enough. The proposed EAI and IAE modules demonstrate that a dynamic, bidirectional interaction between internal and external features is crucial for optimal enhancement.
- Region-wise Enhancement Matters: Treating different regions of an image with distinct degradation patterns separately leads to more natural and effective overall enhancement compared to global processing.
Key Results
- State-of-the-Art Performance: UIERL achieves state-of-the-art results across five benchmark datasets (SQUID, sub-RUIE, sub-UIEB, sub-EUVP, and UISD-1200) based on quantitative metrics like CCF, UIQM, UCIQE, and Edge Intensity.
- Superior Perceptual Quality: The method consistently achieves the highest perceptual scores, indicating visually pleasing and natural-looking enhanced images.
- Improved Color Restoration: UIERL demonstrates strong performance in color correction, as evidenced by low angular reproduction errors on the SQUID-16 dataset.
- Qualitative Improvements: Visual comparisons show UIERL effectively corrects color casts, enhances contrast, and restores details, outperforming existing methods that often suffer from color distortion, over-saturation, or incomplete detail recovery.
Limitations and Future Work
- Computational Cost: While not explicitly detailed as a limitation, the use of multiple related images in the external stage implies a potential increase in computational requirements compared to purely single-image methods.
- Dataset Dependency: The effectiveness of the external stage relies on the availability of related images. In scenarios where only a single, isolated image is available, the external component would not contribute.
- Future Work: The paper suggests that the proposed internal-external learning framework opens new avenues for underwater image enhancement research. Further exploration of more sophisticated interaction mechanisms and potentially real-time applications could be future directions.
Summary updated