Abstract
Recently, various deep denoising methods have been proposed to solve the insufficient feature problem in image denoising. These methods can be mainly classified into two categories: (1) Injecting learnable tensors into denoising backbone to supplement feature, which is effective to some extent but may cause serious over-fitting. (2) Using diverse natural images from large image datasets to synthesize noisy images and pre-train denoising models, which can bring model generalization but require large model size and expensive training costs. To address these issues, this paper proposes Implicit Prompt Learning for Image Denoising (IPLID) method to flexibly generate adaptive prompts without meticulously designing them. Specifically, we first introduce an efficient Linear Prompt (LP) block with ultra-few parameters to produce dynamic prompts for both different stages and samples in denoising procedure. We further propose an efficient Compact Feature Fusion (CFF) block to process previous multi-level prompted denoising feature to reconstruct the denoising images. Finally, to further efficiently and effectively produce satisfactory prompt and denoising performance, a Gradient Accumulation (GA) learning scheme is proposed. Experiments on multiple benchmarks showed that the proposed IPLID achieves competitive results with only 1 percent of pre-trained backbone parameters, outperforming classical denoising methods in both efficiency and quality of restored images.
Introduction
Deep learning has significantly advanced image denoising, yet existing approaches often struggle with a trade-off between model complexity and feature sufficiency. Small models frequently produce over-smoothed, "plastic-looking" images due to limited feature representation. Conversely, large-scale models pre-trained on massive datasets like ImageNet capture rich textures but entail high computational costs, massive parameter counts, and a high risk of over-fitting when fine-tuned on specific denoising tasks. This paper addresses the challenge of leveraging the extensive "knowledge" within large, pre-trained models for denoising without the overhead of full retraining or the pitfalls of over-fitting.
Methodology: Implicit Prompt Learning
The proposed Implicit Prompt Learning for Image Denoising (IPLID) framework avoids full network fine-tuning by freezing a pre-trained backbone (e.g., ResNet-152) and inserting lightweight, learnable modules.
The architecture consists of three primary innovations:
- Linear Prompt (LP) Block: These ultra-lightweight layers are inserted into the encoder to generate "dynamic prompts." These prompts are feature adjustments that adapt to specific noisy inputs and the current stage of the denoising process, effectively acting as noise estimators.
- Compact Feature Fusion (CFF) Block: This specialized decoder component employs dilated (atrous) convolutions to expand the receptive field. This allows the model to capture broader contextual information necessary for reconstruction without a significant increase in parameter count.
- Gradient Accumulation (GA) Learning Scheme: To overcome the difficulty of training with a frozen backbone, the GA scheme applies multi-scale supervision throughout the decoder. This ensures that gradients effectively reach and optimize all learnable blocks.
Experimental Results
IPLID was evaluated on real-world benchmarks including the SIDD (Smartphone Image Denoising Dataset), Nam, and PolyU datasets, which contain authentic sensor noise.
Restoration Quality and Efficiency
IPLID-B (Basic) achieved a PSNR of 40.05 dB on the SIDD dataset, surpassing established models like MIRNet (39.71 dB) and Restormer (40.02 dB). Remarkably, the "Tiny" version (IPLID-T) utilizes only 0.7M trainable parameters—less than 3% of Restormer’s 26.13M—while processing images nearly 11x faster (0.014s vs. 0.153s for a 256x256 image).
Qualitative analysis (see above) highlights IPLID's ability to preserve fine details and complex textures that competing models often blur as noise.
Conclusion
IPLID demonstrates that world-class denoising performance can be achieved by training only 1% of a model's parameters. By utilizing implicit prompts to bridge the gap between frozen pre-trained features and denoising requirements, the method offers a highly efficient and effective solution for low-level vision tasks. Future work may explore more explicit learning of intrinsic noise attributes to further enhance robustness across diverse camera sensors.
Summary updated