PixMamba: Leveraging State Space Models in a Dual-Level Architecture for Underwater Image Enhancement

Wei-Tung Lin, Yong-Xiang Lin, Jyun-Wei Chen, Kai-Lung Hua

Abstract

Underwater Image Enhancement (UIE) is critical for marine research and exploration but hindered by complex color distortions and severe blurring. Recent deep learning-based methods have achieved remarkable results, yet these methods struggle with high computational costs and insufficient global modeling, resulting in locally under- or over- adjusted regions. We present PixMamba, a novel architecture, designed to overcome these challenges by leveraging State Space Models (SSMs) for efficient global dependency modeling. Unlike convolutional neural networks (CNNs) with limited receptive fields and transformer networks with high computational costs, PixMamba efficiently captures global contextual information while maintaining computational efficiency. Our dual-level strategy features the patch-level Efficient Mamba Net (EMNet) for reconstructing enhanced image feature and the pixel-level PixMamba Net (PixNet) to ensure fine-grained feature capturing and global consistency of enhanced image that were previously difficult to obtain. PixMamba achieves state-of-the-art performance across various underwater image datasets and delivers visually superior results.

Introduction

Capturing clear images underwater is notoriously difficult due to wavelength-dependent light absorption and scattering, which result in severe color distortion, low contrast, and blurring. While deep learning has improved Underwater Image Enhancement (UIE), existing architectures face a significant trade-off. Convolutional Neural Networks (CNNs) are computationally efficient but have limited receptive fields, making it difficult to maintain global color consistency. Conversely, Transformers excel at global modeling but suffer from quadratic computational complexity relative to image resolution, making them memory-intensive for high-resolution tasks.

PixMamba bridges this gap by utilizing State Space Models (SSMs), specifically the Mamba architecture. Mamba provides the global receptive field of a Transformer but with the linear scaling and efficiency of a CNN. By combining a patch-level network for texture reconstruction with a pixel-level network for fine-grained consistency, PixMamba achieves state-of-the-art results with significantly higher computational efficiency than Transformer-based models.

Architecture and Methodology

The core of PixMamba is a dual-level strategy that processes images through two parallel paths to capture both coarse and fine-grained features.

The PixMamba architecture, featuring the dual-level EMNet and PixNet paths along with the Efficient Mamba Block (EMB) and scanning mechanism.

Efficient Mamba Net (EMNet)

EMNet is a patch-level network based on a U-Net style architecture. It utilizes Efficient Mamba Blocks (EMB) to model dependencies between image patches. A key innovation is the Mamba Upsampling Block (MUB), which integrates an SSM mechanism before upsampling. This allows the model to intelligently prioritize which features to preserve, reducing the detail loss common in standard U-Net architectures.

PixMamba Net (PixNet)

Operating in parallel to EMNet, PixNet processes the image at the individual pixel level. To maintain spatial awareness while treating pixels as a sequence, the authors introduced Block-wise Positional Embedding (BPE). This allows the model to capture fine-grained details and ensure global color and clarity consistency, which patch-based methods often struggle to maintain.

Efficient SS2D (ESS2D)

To further optimize performance, the authors developed Efficient SS2D (ESS2D). This component simplifies the standard 2D selective scan—which typically scans in four directions—to reduce computational overhead while maintaining the effectiveness of global dependency modeling.

Experimental Results

The model was evaluated on two primary public benchmarks: the Underwater Image Enhancement Benchmark (UIEB) and the Underwater Color Cast Set (UCCS).

Quantitative Performance

PixMamba achieved state-of-the-art (SOTA) performance across multiple metrics. On the UIEB T90 dataset, it reached a PSNR of 23.587 and an SSIM of 0.921, outperforming established models like Ucolor and Semi-UIR.

In terms of efficiency, PixMamba demonstrated a significant advantage. On the C60 dataset, it achieved better UIQM (Underwater Image Quality Measure) scores than the previous SOTA (WaterMamba) while maintaining a similar computational footprint of approximately 7.6G FLOPs. For comparison, the Transformer-based Restormer requires 140.99G FLOPs for lower performance.

Qualitative Analysis

Visual results indicate that PixMamba is particularly effective at removing the "haze" effect and restoring natural-looking textures in high-detail areas, such as coral reefs and fish scales. The dual-path approach ensures that while textures are reconstructed at the patch level, global color saturation and clarity are maintained at the pixel level, avoiding the artifacts or over-smoothing common in other models.

Conclusion and Future Work

PixMamba demonstrates the power of linear-complexity State Space Models for high-resolution image restoration tasks. By leveraging a dual-level architecture, it successfully balances global context modeling with fine-grained detail preservation.

Future work may focus on further optimizing the 8.68M parameter architecture for real-time deployment on low-power underwater drones. Additionally, extending the Mamba-based temporal modeling to underwater video sequences and testing robustness in extreme, highly turbid deep-sea environments remain promising avenues for research.

Summary updated

Abstract

Underwater Image Enhancement (UIE) is critical for marine research and exploration but hindered by complex color distortions and severe blurring. Recent deep learning-based methods have achieved remarkable results, yet these methods struggle with high computational costs and insufficient global modeling, resulting in locally under- or over- adjusted regions. We present PixMamba, a novel architecture, designed to overcome these challenges by leveraging State Space Models (SSMs) for efficient global dependency modeling. Unlike convolutional neural networks (CNNs) with limited receptive fields and transformer networks with high computational costs, PixMamba efficiently captures global contextual information while maintaining computational efficiency. Our dual-level strategy features the patch-level Efficient Mamba Net (EMNet) for reconstructing enhanced image feature and the pixel-level PixMamba Net (PixNet) to ensure fine-grained feature capturing and global consistency of enhanced image that were previously difficult to obtain. PixMamba achieves state-of-the-art performance across various underwater image datasets and delivers visually superior results. Code is available at: https://github.com/weitunglin/pixmamba.
BibTeX
@inproceedings{lin2022pixmamba,
  author = {Wei-Tung Lin and Yong-Xiang Lin and Jyun-Wei Chen and Kai-Lung Hua},
  title = {{PixMamba: Leveraging State Space Models in a Dual-Level Architecture for Underwater Image Enhancement}},
  booktitle = {ACCV},
  year = {2022},
  url = {https://openreview.net/forum?id=7LDofhR3Sy},
}
Figure