Phaseformer: Phase-based Attention Mechanism for Underwater Image Restoration and Beyond

Md Raqib Khan, Anshul Negi, Ashutosh Kulkarni, Shruti S. Phutke, Santosh Kumar Vipparthi, Subrahmanyam Murala

TL;DR

Phaseformer is a lightweight transformer network (1.77M parameters) designed for efficient underwater image restoration by exploiting a key physical insight: underwater degradation severely distorts image amplitude (color and contrast) but leaves structural phase information relatively intact. By introducing phase-based self-attention and optimized skip connections, Phaseformer achieves state-of-the-art restoration quality with a fraction of the computational cost of competing models.

What Problem Does This Paper Address?

Underwater imaging is notoriously difficult due to light refraction, absorption, and scattering by water molecules and suspended particles. This results in severe color casts (often heavy blue or green tints), haziness, blurriness, and low contrast. For autonomous underwater vehicles (AUVs) and marine robotics relying on computer vision, this degradation severely degrades performance in tasks like object detection and mapping.

While deep learning and transformer-based models have advanced underwater image enhancement (UIE), they are typically computationally heavy, making them impractical for real-time deployment on autonomous edge devices. Furthermore, standard architectures often inadvertently propagate noise and degraded features through direct skip connections. Phaseformer addresses this dual challenge by designing a highly efficient, structure-focused architecture.

Methods

The core innovation of Phaseformer is its reliance on frequency-domain phase information to guide spatial attention. The architecture is built as a U-Net variant utilizing two novel components:

  1. Phase-based Multi-head Self-Attention (PMSA): Inside the Phase-based Transformer Blocks (PBTB), the network uses a Phase Extraction Module (PEM). By applying a Fast Fourier Transform (FFT) to the feature maps, setting the amplitude to a constant, and computing the Inverse FFT (IFFT), the network isolates pure structural phase features. The queries ($Q$) and keys ($K$) are derived from these clean phase features, ensuring the attention mechanism calculates pixel interactions based on stable structural layouts rather than corrupted colors or noise.
  2. Optimized Phase Attention Block (OPAB): Instead of using standard, direct skip connections that pass degraded encoder features straight to the decoder, Phaseformer routes them through the OPAB. This block applies global average pooling and an adaptive 1D convolution whose kernel size dynamically scales with the channel depth, preventing feature over-smoothing or under-smoothing.
Figure 2. Architectural schematic of the proposed underwater image enhancement network. The network comprises of Phase-based Transformer Block, Optimized Phase Attention Block, and dynamic weight assignment to the loss functions. The proposed phase- based transformer block captures local and non-local structural information using less attenuated phase-based queries and keys. Optimized Phase Attention block is proposed for propagating prominent attentive features from the encoder to the respective decoder. The dynamic weight (Ω) assignment is proposed for efficient optimization of the network while training.

Additionally, the authors introduce an Adaptive Weight Assignment strategy. Instead of manually tuning the weights of the multiple loss functions used (Charbonnier, Gradient, MS-SSIM, and Perceptual losses), the network treats these weights as learnable tensors that optimize automatically during training.

Datasets

The model was trained and evaluated across a comprehensive suite of synthetic and real-world datasets:

  • Synthetic: UIEB (Underwater Image Enhancement Benchmark—800 training pairs) and UFO-120 (1500 training pairs).
  • Real-World: U45 (45 challenging real-world images), UCCS (Underwater Color Cast Set), and SQUID (Stereo Quantitative Underwater Image Dataset).
  • Extensions: The authors also tested the model's versatility on the LOL (Low-Light) dataset for low-light image enhancement.

Key Insights and Ideas

  • The Resilience of Phase: Using t-SNE dimensionality reduction, the authors demonstrate that clean and degraded underwater images form completely distinct clusters when analyzing amplitude, but heavily overlap when analyzing phase. This confirms that phase information acts as a reliable, degradation-resistant skeleton of the image.
Figure 1. t-SNE visualization of the Amplitude and Phase of clean and degraded images. The separate clusters for clean and degraded amplitude show that there is more effect of degradation on ampli- tude content as compared to phase content which has overlapping clusters for clean and noisy images.
  • Phase-Guided Attention Filters Noise: By computing self-attention maps purely from phase data, the transformer avoids getting "distracted" by severe color shifts or localized haziness, allowing it to restore global color consistency more effectively.
  • Smart Skip Connections: Filtering skip connections through a phase-aware attention block ensures that only prominent, clean structural boundaries are passed to the decoder, actively filtering out encoder-level noise.

Key Results

Phaseformer delivers state-of-the-art performance while remaining remarkably lightweight:

  • Restoration Quality: On the synthetic UIEB dataset, Phaseformer achieved a PSNR of 25.98 dB and SSIM of 0.928, outperforming recent SOTA models like Spectroformer (24.96 dB / 0.917) and Semi-UIR (24.59 dB / 0.901).
  • Computational Efficiency: The model contains only 1.77M parameters and requires 13.0 GFLOPs. In comparison, the popular U-shape transformer uses 65.6M parameters and 66.2 GFLOPs. Phaseformer processes images at an impressive 0.034 seconds per image, making it highly viable for real-time robotic applications.
  • Downstream Boost: Using Phaseformer as a pre-processing step significantly improved accuracy for downstream tasks like object detection (via YOLO-v3) and monocular depth estimation.

Limitations and Future Work

While Phaseformer excels at correcting severe color casts and restoring contrast, the authors note a limitation in handling extremely muddy or heavily blurred water conditions. The current architecture focuses heavily on structural and color restoration, leaving room for improvement in dedicated image deblurring. The authors plan to target specialized deblurring mechanisms in future iterations of the network.

Summary updated

Abstract

Quality degradation is observed in underwater images due to the effects of light refraction and absorption by water, leading to issues like color cast, haziness, and limited visibility. This degradation negatively affects the performance of autonomous underwater vehicles used in marine applications. To address these challenges, we propose a lightweight phase-based transformer network with 1.77M parameters for underwater image restoration (UIR). Our approach focuses on effectively extracting non-contaminated features using a phase-based self-attention mechanism. We also introduce an optimized phase attention block to restore structural information by propagating prominent attentive features from the input. We evaluate our method on both synthetic (UIEB, UFO-120) and real-world (UIEB, U45, UCCS, SQUID) underwater image datasets. Additionally, we demonstrate its effectiveness for low-light image enhancement using the LOL dataset. Through extensive ablation studies and comparative analysis, it is clear that the proposed approach outperforms existing state-of-the-art (SOTA) methods.
BibTeX
@inproceedings{khan2025phaseformer,
  author = {MD Raqib Khan and Anshul Negi and Ashutosh Kulkarni and Shruti S. Phutke and Santosh Kumar Vipparthi and Subrahmanyam Murala},
  title = {{Phaseformer: Phase-based Attention Mechanism for Underwater Image Restoration and Beyond}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2025},
}
Figure