StruNet: Perceptual and low‐rank regularized transformer for medical image denoising

Yuhui Ma, Qifeng Yan, Yonghuai Liu, Jiang Liu, Jiong Zhang, Yitian ZhaoMedical 2023 · DOI

Abstract

The results demonstrate that the proposed architecture yields a promising performance of suppressing multiform noise artifacts existing in different imaging modalities.

Introduction

Medical imaging is a critical tool for diagnosis, yet it frequently involves a trade-off between patient safety (such as minimizing radiation exposure in CT) and image quality. This trade-off often introduces modality-specific noise: "mottle" in Computed Tomography (CT), "speckle" in Optical Coherence Tomography (OCT), and "stripe" artifacts in OCT Angiography (OCTA).

Visual comparison of low-quality and high-quality medical images across CT, OCT, and OCTA modalities showing characteristic noise patterns.

While Deep Learning (DL) has significantly advanced denoising, standard Convolutional Neural Networks (CNNs) often struggle with textural modeling due to their limited receptive fields. This makes it difficult to capture complex noise patterns without sacrificing fine anatomical details. Transformers offer a solution through global attention mechanisms, but they can be computationally intensive and prone to over-smoothing critical diagnostic features. To address these challenges, the authors propose StruNet (Swin transformer-based residual u-shape Network), a hybrid framework designed to preserve structural integrity while effectively removing noise.

Methodology

StruNet introduces a novel architecture that combines the global modeling capabilities of Swin Transformers with the detail-preserving nature of residual blocks. The framework is built upon a U-shape network with several key innovations:

Hybrid Encoder-Decoder Architecture

Instead of relying solely on one architecture type, StruNet utilizes parallel paths within its encoder and decoder layers.

  • Swin Transformer Modules: These modules use shifted windows to learn hierarchical representations of noise across different scales with linear computational complexity, allowing the model to understand the broader context of noise artifacts.
  • Residual Blocks: These act as a high-frequency "shortcut," ensuring that fine anatomical structures—like retinal layers or small vessels—are preserved and not lost during the Transformer's global processing.
The proposed StruNet architecture featuring parallel encoder-decoder blocks with Swin Transformers and residual connections.

Triple-Threat Loss Function

To ensure the denoised images are both mathematically accurate and visually realistic, the model employs a multi-faceted loss function:

  1. $L_1$ Loss: Measures pixel-to-pixel differences between the denoised and ground-truth images.
  2. Perceptual Loss: Utilizes a pre-trained VGG-19 network to ensure the denoised image maintains high-level features that align with human visual perception.
  3. Low-Rank Regularization: Drawing from classical signal processing, this term penalizes high-rank "noise" components. Since clean medical images generally exhibit low-rank characteristics (mathematical redundancy), this constraint guides the network toward producing cleaner structural patterns.

Experiments and Results

The model was validated across three distinct modalities to demonstrate its versatility: CT scans from the AAPM Low Dose CT Grand Challenge, OCT B-scans with simulated low-bit-depth noise, and OCTA images with synthetic stripe artifacts.

StruNet consistently outperformed traditional filters like BM3D and modern DL models such as RED-CNN and CycleGAN:

  • Low-Dose CT: Achieved a PSNR (Peak Signal-to-Noise Ratio) of 32.902, a significant improvement over the noisy baseline (18.177) and the competing RED-CNN (30.334).
  • OCT Reconstruction: Improved PSNR by 26.91% and SSIM (Structural Similarity) by 27.40% compared to noisy low-bit-depth images.
  • OCTA Destriping: Effectively removed motion-induced stripes, achieving the highest PSNR (30.229) and SSIM (0.958) among all tested methods.

Beyond standard metrics, the authors demonstrated that StruNet improves performance in downstream clinical tasks, such as vessel and choroid segmentation, indicating that the denoised images are more machine-readable and clinically useful.

Discussion and Conclusion

The success of StruNet lies in its ability to balance global context with local detail. By integrating low-rank regularization as a prior, the model provides a mathematical constraint that helps identify noise without requiring explicit labels for every artifact type. Furthermore, with an inference time of approximately 0.128 seconds per image, the model is viable for near-real-time clinical applications.

A current limitation is the reliance on paired data for training, which can be difficult to obtain in real-world clinical settings. Future work will focus on semi-supervised or blind denoising strategies to allow the model to learn from unpaired, real-world noisy data, further bridging the gap between laboratory benchmarks and bedside reality.

Summary updated

Abstract

The results demonstrate that the proposed architecture yields a promising performance of suppressing multiform noise artifacts existing in different imaging modalities.
BibTeX
@inproceedings{ma2023strunet,
  author = {Yuhui Ma and Qifeng Yan and Yonghuai Liu and Jiang Liu and Jiong Zhang and Yitian Zhao},
  title = {{StruNet: Perceptual and low‐rank regularized transformer for medical image denoising}},
  booktitle = {Medical},
  year = {2023},
  doi = {10.1002/mp.16550},
}
Figure