Abstract
Since Deep Convolutional Neural Networks (DCNNs) and Vision Transformer perform well in learning generalizable image priors from large-scale data, these models have been widely used in image denoising tasks. However, vanilla DCNNs and Transformer suffer from two problems. First, the vanilla DCNNs and Transformer only accumulate the output along the channel axis, ignoring the internal relationship among channels. This results in the severely inadequate color structure representation retrieved from color images. Secondly, the DCNNs or Transformer-based image denoising models usually have a large number of parameters, high computational complexity, and slow inference speed. To resolve these issues, this paper proposes a highly-efficient Quaternion Transformer (QFormer) for image denoising. Specifically, the proposed Quaternion Transformer Block (QTB) simplifies the typical Transformer from a multi-branch structure to an elaborately sequential structure mainly with quaternion transformations, to alternately capture both long-range dependencies and local contextual features with color structure information. Furthermore, the proposed QTB can also avoid considerable element-wise multiplications of computing the self-attention matrices. Thus, our QTB can significantly reduce the computational complexity and its sequential structure can further improve the practical inference speed. Comprehensive experiments demonstrate that the proposed QFormer produces state-of-the-art results in both denoising performance and efficiency.
Introduction
Modern image denoising architectures, primarily based on Deep Convolutional Neural Networks (DCNNs) and Transformers, often treat color images as stacks of independent monochromatic channels. This approach ignores the structural interrelationships between color channels, frequently leading to color distortion in the restored images. Additionally, high-performance Transformers are often computationally expensive due to self-attention mechanisms that scale quadratically with image resolution.
This paper introduces QFormer, a novel architecture designed to be both "color-aware" and "computationally lean" by integrating quaternion algebra into the Transformer framework.
Methodology
The core of the QFormer architecture is the Quaternion Transformer Block (QTB). The overall pipeline transforms color noisy images into the quaternion domain for processing, as illustrated in the following diagram.
Quaternion Representation and Processing
QFormer represents RGB pixels as "pure quaternions" ($I = Ri + Gj + Bk$). This mathematical framework allows the network to process all three color channels simultaneously, inherently preserving the inter-channel relationships.
The Quaternion Transformer Block (QTB)
The QTB replaces the standard multi-branch Transformer structure with a streamlined sequential design. It utilizes quaternion fully connected (Q-FC) layers to extract global features and quaternion convolutions (Q-Conv) to capture local context.
To maintain efficiency, QTB avoids the massive computational cost of standard self-attention matrices by using element-wise addition to aggregate global and local features. This captures long-range dependencies without the $O(N^2)$ bottleneck. Furthermore, the authors incorporate short skip connections to ensure "identity mapping," which is crucial for preserving high-frequency details that might otherwise be lost during quaternion transformations.
Experimental Evaluation
The proposed QFormer was evaluated on three major real-world noisy image benchmarks: the Smartphone Image Denoising Dataset (SIDD), PolyU, and Nam.
Performance and Efficiency
QFormer demonstrated superior performance compared to 14 state-of-the-art denoising methods:
- Parameter Reduction: QFormer-T (Tiny) is approximately 22.8 times smaller than the Restormer model while achieving comparable results.
- Denoising Accuracy: QFormer-B (Base) achieved a PSNR of 40.18 dB on the SIDD dataset, outperforming heavyweights like MIRNet.
- Inference Speed: Ablation studies showed that QFormer's sequential structure significantly improves practical inference speed and reduces FLOPs compared to traditional UNet-based quaternion models.
- Visual Quality: Qualitative results indicate that QFormer is particularly effective at preserving fine contours and edges, avoiding the blurring artifacts common in other models.
Conclusion
QFormer presents a highly efficient alternative to standard Transformer architectures for image denoising. By leveraging quaternion algebra, it achieves superior color structure representation and significant computational savings. Future research may explore extending this architecture to other complex domains, such as multispectral imagery or other image restoration tasks like super-resolution.
Summary updated