Abstract
Discriminative learning approaches have achieved a convincing trade-off between image quality and computational efficiency in restoration tasks. However, these methods typically require separate training for each specific task and condition, leading to a "brittle" specialization. We present Discriminative Transfer Learning (DTL), a framework that incorporates formal proximal optimization with discriminative learning. DTL allows for a single-pass training that can be reused across various restoration problems—such as denoising, deblurring, and inpainting—while maintaining the speed of specialized discriminative models.
Introduction: Bridging the Gap
In computer vision, image restoration has long been divided between two philosophies. Generative/classical models are flexible and modular but computationally expensive. Conversely, discriminative models (like early neural networks) are fast but specialized; they often require a unique model for every task or noise level. DTL offers a middle ground: the speed of discriminative learning with the "plug-and-play" modularity of classical optimization.
The Problem: Specialized vs. Versatile Models
Traditional discriminative models are highly specialized. A model trained for denoising at a specific noise level ($\sigma=15$) often fails when applied to $\sigma=25$ or a different task like deblurring. This necessitates storing dozens of models for different scenarios, which is impractical for real-world applications like mobile photography. The challenge lies in building a model as fast as a specialized network but as versatile as a general-purpose optimization algorithm.
Methodology: Decoupling Physics from Statistics
The core innovation of DTL is the decoupling of the image restoration objective into two alternating subproblems using Half-Quadratic Splitting (HQS):
- The Data Proximal Operator: This handles the task-specific physics (e.g., a specific blur kernel). It is mathematically well-defined and does not require learning.
- The Prior Proximal Operator: This handles the "naturalness" of the image. The authors realized that while the data operator changes per task, the prior operator can act as a universal Gaussian denoiser.
Instead of a slow handcrafted prior, DTL uses a modified Trainable Nonlinear Reaction Diffusion (TRD) model as the prior operator. By using a recurrent architecture, the same learned parameters are reused across multiple iterations, keeping the model compact and ensuring convergence.
Key Results and Performance
DTL demonstrates significant robustness compared to specialized models. While specialized TRD models see a sharp decline in performance when test noise deviates from training noise, DTL maintains stable and superior PSNR across a range of conditions.
- Versatility: A single DTL model can be transferred to solve untrained tasks like inpainting or deblurring by simply swapping the data operator at test time.
- Competitive Quality: In denoising, DTL achieves PSNR results (~31.00 dB) comparable to state-of-the-art specialized methods like BM3D. In non-blind deconvolution, it achieved an average PSNR of 34.34 dB on the Levin benchmark, outperforming previous methods like CSF (33.48 dB).
- Efficiency: Implemented in Halide, DTL processes a $512 \times 512$ image in just 0.26 seconds, compared to 111 seconds for optimization-based methods like EPLL.
Conclusion
Discriminative Transfer Learning successfully bridges the gap between modularity and speed. By embedding a discriminative prior within a formal optimization loop, DTL provides a "universal" restorer that is both fast and flexible. Future work may extend this approach to deeper architectures, potentially leading to even more powerful and compact universal image priors.
Summary updated