Semantic-aware Texture-Structure Feature Collaboration for Underwater Image Enhancement

Di Wang, Long Ma, Risheng Liu, Xin Fan

Introduction

Underwater imaging is a critical component of marine exploration, robotics, and environmental monitoring. However, the medium presents significant optical challenges: light absorption and scattering cause severe color casts, low contrast, and blurred details. While deep learning has advanced the field of underwater image enhancement (UIE), many existing models suffer from "structural amnesia." They often prioritize pixel-level color correction at the expense of fine textures and large-scale structural integrity, making the resulting images less effective for high-level computer vision tasks like object tracking or robotic navigation.

This paper introduces a novel framework that addresses these limitations by leveraging high-level semantic features from a pretrained model (VGG16) to guide low-level restoration. By explicitly separating and refining "texture" and "structure" features, the proposed method achieves state-of-the-art results that are both visually pleasing and machine-readable.

Methodology

The proposed architecture is built upon three primary components designed to collaborate across different feature hierarchies.

1. UNet-like Base Network

The core of the system is a standard encoder-decoder reconstruction network. To maximize information utilization across various receptive fields, the authors incorporate a Pyramid Context Block at the bottleneck. This ensures that the network captures global context before the synthesis phase begins.

2. Semantic-aware Feature Aggregation Network (SFANet)

The SFANet serves as the "semantic brain" of the framework. It utilizes a pretrained VGG16 backbone to extract hierarchical features. The authors categorize these into two distinct types:

  • Texture Features: Extracted from shallow layers to capture fine details like sand grains or fish scales.
  • Structure Features: Extracted from deep layers to capture the global geometry and shapes of objects within the scene.

These features are processed through a Multi-path Contextual Feature Refinement Module (CFRM), which uses parallel convolution paths (3x3, 5x5, and 7x7 kernels) to capture multi-scale correlations before aggregation.

3. Feature Dominative Network (FDNet)

Simply concatenating semantic features with the base network is often insufficient. The FDNet acts as a modulation layer, using a Channel Transformation Layer (CTL) to adapt semantic features to the specific patterns required by the decoder. This ensures that the semantic "hints" are properly scaled and integrated during the image synthesis process.

Experimental Evaluation

The model was evaluated against several benchmarks, including the Real-world Underwater Image Enhancement (UIEB) dataset, the Large-scale Underwater Image (EUVP) dataset, and the Underwater Salient Object Detection (USOD) dataset.

Quantitative and Qualitative Performance

The framework demonstrated clear superiority over existing methods like Ucolor and WaterNet. On the UIEB dataset, the model achieved a PSNR of 22.45 dB, significantly higher than Ucolor (20.62 dB). Similar leads were observed on the EUVP dataset (23.23 dB). Beyond metrics, the qualitative results showed better color fidelity and sharper structural boundaries.

Downstream Task Impact

A key contribution of this research is the validation of enhancement for "machine vision." In salient object detection (SOD) tests, images enhanced by this method led to a much higher F-measure (0.854) and lower Mean Absolute Error (0.080) compared to other techniques. This proves that the semantic-aware approach successfully restores the structural information necessary for automated systems to understand underwater scenes.

Discussion and Conclusion

The success of this framework highlights two critical insights in underwater vision:

  1. Hierarchy Matters: Using a full hierarchy of features (texture vs. structure) is more effective than using flat semantic priors or segmentation masks.
  2. Decoder-Side Injection: Ablation studies confirmed that embedding semantic features into the decoder is more effective than the encoder, suggesting that semantic guidance is most valuable during the reconstruction phase.

While the model is computationally efficient and runs faster than many SOTA models on a standard GPU, future work may focus on training backbones specifically on unlabeled underwater data to further reduce the dependency on terrestrial-trained models like VGG16. This approach paves the way for more robust, real-time enhancement systems for underwater robotics.

Summary updated

Abstract

Underwater image enhancement has become an attractive topic as a significant technology in marine engineering and aquatic robotics. However, the limited number of datasets and imperfect hand-crafted ground truth weaken its robustness to unseen scenarios, and hamper the application to high-level vision tasks. To address the above limitations, we develop an efficient and compact enhancement network in collaboration with a high-level semantic-aware pretrained model, aiming to exploit its hierarchical feature representation as an auxiliary for the low-level underwater image enhancement. Specifically, we tend to characterize the shallow layer features as textures while the deep layer features as structures in the semantic-aware model, and propose a multi-path Contextual Feature Refinement Module (CFRM) to refine features in multiple scales and model the correlation between different features. In addition, a feature dominative network is devised to perform channel-wise modulation on the aggregated texture and structure features for the adaptation to different feature patterns of the enhancement network. Extensive experiments on benchmarks demonstrate that the proposed algorithm achieves more appealing results and outperforms state-of-the-art methods by large margins. We also apply the proposed algorithm to the underwater salient object detection task to reveal the favorable semantic-aware ability for high-level vision tasks. The code is available at STSC.
BibTeX
@inproceedings{wang2022semantic,
  author = {Di Wang and Long Ma and Risheng Liu and Xin Fan},
  title = {{Semantic-aware Texture-Structure Feature Collaboration for Underwater Image Enhancement}},
  booktitle = {IEEE International Conference on Robotics and Automation},
  year = {2022},
  url = {https://openreview.net/forum?id=IrZoWIzJDd},
}
Figure