Introduction
Image restoration tasks, such as denoising and JPEG artifact removal, face a significant challenge: the variability of degradation levels. In real-world scenarios, the intensity of noise or compression is often unknown and non-uniform. Traditional deep learning approaches typically employ fixed-depth architectures, applying a constant amount of computation regardless of the image's corruption level. This lack of adaptivity often leads to poor generalization, where models trained on specific noise levels fail when encountering more severe degradation.
To address this, we introduce the Dynamically Unfolding Recurrent Restorer (DURR). By framing image restoration as a "moving endpoint" control problem, DURR adaptively determines the optimal number of iterations required for each specific image, allowing for a flexible and efficient restoration process.
Moving Endpoint Control Framework
The DURR framework treats image restoration as a dynamic system where the "terminal time" (the number of processing steps) is a variable to be optimized. The system consists of two primary modules:
- Restoration Unit (RU): This unit is a convolutional Recurrent Neural Network (RNN) with a U-Net-like architecture. It is designed to mimic nonlinear diffusion processes, which are effective at smoothing noise while preserving critical edge information. As an RNN, the RU can be iteratively applied to the image, progressively refining the result.
- Policy Unit (PU): Acting as the controller, the PU decides when to stop the restoration process. It utilizes ResNet blocks and an LSTM cell to analyze the current state of the image at each iteration. The PU outputs a binary decision: "Again" (continue processing) or "Done" (terminate and output the result).
Training Strategy
The model is trained in a two-stage process to ensure both restoration quality and decision-making accuracy:
- Stage 1: Restoration Unit Optimization. The RU is trained using a "refined policy" where optimal iteration counts are pre-determined for various noise levels. This ensures the RU learns an effective iterative mapping.
- Stage 2: Policy Unit Reinforcement Learning. The PU is trained using Deep Q-learning. The reward function is tied to the improvement in Peak Signal-to-Noise Ratio (PSNR). The agent receives a positive reward for increasing image quality and a penalty for each additional iteration, encouraging efficiency.
Experimental Results
DURR was evaluated on standard benchmarks including BSD68 for denoising and LIVE1 for JPEG deblocking. The results demonstrate significant advantages over state-of-the-art fixed-depth models like DnCNN.
Generalization to Extreme Degradation
One of the most striking features of DURR is its ability to generalize to degradation levels far beyond those seen during training. For instance, when faced with extreme Gaussian noise ($\sigma = 95$), DURR maintains high visual fidelity and a PSNR of 24.94 dB. In contrast, fixed-depth models like DnCNN often collapse, producing results with significantly lower PSNR (e.g., 14.46 dB) and severe artifacts.
Efficiency and Parameter Count
Despite its iterative nature, DURR is highly parameter-efficient. It achieves superior performance compared to DnCNN-B while using fewer parameters. Furthermore, the "Doppio-DURR" variant demonstrates that reducing network width while allowing for more iterations can lead to faster inference times without sacrificing quality.
Conclusion
The Dynamically Unfolding Recurrent Restorer represents a paradigm shift from fixed-depth to dynamic-depth architectures in image restoration. By leveraging moving endpoint control and reinforcement learning, DURR provides a robust solution that adapts to unknown degradation levels. Its exceptional generalization capabilities and parameter efficiency make it a promising framework for various ill-posed inverse problems in computer vision, such as deraining and deblurring.
Summary updated