Abstract
Image denoising remains a fundamental challenge in computer vision, particularly in capturing inter-channel correlations and maintaining computational efficiency. Traditional methods often process color channels independently, leading to color distortions. We present QFormer, a novel Transformer architecture based on quaternion algebra. By representing RGB images as quaternion matrices, QFormer preserves complex inter-channel relationships. The core component, the Quaternion Transformer Block (QTB), employs a sequential design to extract global and local features without the quadratic complexity of standard self-attention. Experimental results on SIDD, PolyU, and Nam datasets demonstrate that QFormer achieves state-of-the-art performance with significantly fewer parameters and faster inference speeds than traditional real-valued Transformers.
Introduction
Digital image denoising aims to recover clean images from noisy observations. Despite the success of deep learning models, two major issues persist: the neglect of inter-channel correlations (monochromatic processing) and the quadratic computational complexity of standard Vision Transformers (ViTs). Most modern denoising models process the Red, Green, and Blue channels independently or simply sum their outputs, ignoring the structural relationship between colors. Furthermore, the self-attention mechanism in ViTs is often too heavy for high-resolution image tasks, where complexity grows quadratically with the number of pixels.
The QFormer Architecture
QFormer addresses these challenges by adopting a quaternion-based framework. Instead of treating an image as a standard 3D tensor, it is represented using quaternions—a number system extending complex numbers. An RGB pixel is mapped to a "pure quaternion" ($Ri + Gj + Bk$), allowing the network to process all three color channels as a single mathematical unit.
As shown in Figure 1, the pipeline involves decomposing the noisy image into channels, applying a quaternion rotation matrix, and passing the data through an Encoder-Decoder structure powered by the QFormer Network. This approach ensures the model understands the rotation and scaling relationships between channels, preventing common artifacts like color bleeding.
Quaternion Transformer Block (QTB)
The core innovation of QFormer is the Quaternion Transformer Block (QTB). Unlike the multi-branch structure of standard Transformers, the QTB uses an elaborately sequential design to aggregate features efficiently.
The QTB (Figure 2) alternates between extracting global features (using quaternion fully connected layers) and local features (using quaternion convolutions). By using element-wise addition instead of heavy matrix multiplications for attention, QTB significantly reduces computational overhead. Additionally, the authors identified that primitive quaternion operators do not naturally act as identity mappings. To ensure high-frequency details like textures are preserved, specialized short skip connections were incorporated to alleviate this mathematical bottleneck.
Experimental Results
QFormer was evaluated on three prominent real-world noisy image datasets: SIDD (Smartphone Image Denoising Dataset), PolyU, and Nam. These datasets provide a rigorous test for denoising models due to the complexity of actual sensor interference compared to synthetic noise.
- State-of-the-Art Performance: QFormer consistently outperformed 14 other state-of-the-art methods, including Restormer and Uformer. On the PolyU dataset, QFormer-S improved the Peak Signal-to-Noise Ratio (PSNR) by 0.26 dB over previous leaders.
- Efficiency and Speed: The architecture is remarkably lightweight. The "Tiny" version (QFormer-T) is approximately 22.8 times smaller in terms of parameters than Uformer while delivering comparable or better results. Inference tests showed a reduction in processing time by over 70% compared to standard quaternion-based UNet baselines.
Conclusion
QFormer demonstrates that quaternion algebra can effectively bridge the gap between color-aware processing and computational efficiency in image denoising. By rethinking the Transformer block as a sequential quaternion operator, the model achieves state-of-the-art results on real-world datasets while remaining lightweight enough for practical applications. This work encourages further exploration of non-real-valued networks for sophisticated color processing in low-level vision tasks.
Summary updated