Introduction
Image super-resolution (SR) is a fundamental yet ill-posed inverse problem in computer vision, where the goal is to reconstruct a high-resolution (HR) image from its low-resolution (LR) counterpart. Traditional methods, often optimized for pixel-wise metrics like Peak Signal-to-Noise Ratio (PSNR), frequently produce blurry results by averaging the manifold of possible HR solutions. While generative models such as Diffusion Probabilistic Models (DPMs) have demonstrated the ability to synthesize sharp, realistic details, they are typically hindered by high computational costs during inference and a tendency toward structural or color inconsistencies relative to the input LR image.
To address these limitations, this paper introduces the Efficient Conditional Diffusion Model with Probability Flow Sampling (ECDP). This framework enhances diffusion-based SR by optimizing for both inference speed and perceptual fidelity through a continuous-time formulation and a novel training objective.
Methodology
ECDP introduces three primary technical innovations designed to bridge the gap between generative quality and practical efficiency.
Continuous-Time SDE and Probability Flow
Rather than relying on discrete diffusion steps, ECDP models the image corruption process as a continuous-time Stochastic Differential Equation (SDE). By leveraging the corresponding Probability Flow Ordinary Differential Equation (ODE), the model can utilize efficient numerical solvers, such as Runge-Kutta methods. This transition from stochastic sampling to deterministic probability flow significantly reduces the number of iterations required to generate high-quality images.
Hybrid Parametrization
Standard diffusion models typically predict either the added noise ($\epsilon$-parametrization) or the clean image ($x_0$-parametrization). The authors observe that $\epsilon$-prediction excels at capturing fine details at low noise levels but struggles with global structure at high noise levels, leading to color shifting. ECDP employs a hybrid parametrization that dynamically interpolates between these two targets based on the current noise scale, ensuring structural consistency and natural color reproduction.
Adjoint-Method Perceptual Loss
To improve visual appeal, the framework incorporates a VGG-based perceptual loss ($L_{quality}$). Training with such a loss through an ODE solver is typically memory-prohibitive. ECDP circumvents this by using the adjoint method, which allows for gradient computation with constant memory cost, regardless of the number of sampling steps. This enables the model to be tuned for human-perceived quality directly within the diffusion training loop.
Experimental Evaluation
The model was evaluated on standard benchmarks for both general and domain-specific super-resolution:
- General SR (4×): Evaluated on DIV2K (augmented with Flickr2K) and ImageNet.
- Face SR (8×): Evaluated on CelebA to test the reconstruction of fine facial textures.
Key Results
- Inference Speed: ECDP achieves a massive reduction in latency. It generates a 256×256 image in approximately 2.0 seconds, outperforming previous diffusion models like SR3 (83.1s) and IR-SDE (6.2s).
- Perceptual Fidelity: On the DIV2K dataset, ECDP achieved an LPIPS score of 0.108, significantly better than SR3 (0.175) and SRDiff (0.136).
- Consistency: Qualitative analysis shows that ECDP effectively eliminates common diffusion artifacts, such as grid patterns and color drifts, producing sharper architectural lines and more realistic skin and hair textures.
Discussion and Conclusion
ECDP represents a significant step forward in making diffusion-based super-resolution viable for real-world applications. By anchoring the forward process to the mean of the upscaled LR image and utilizing hybrid parametrization, the model maintains high fidelity to the source material while generating plausible high-frequency details.
While the 2-second inference time is a breakthrough for DPMs, it remains slower than feed-forward GAN or CNN architectures. Future work may focus on further reducing this latency and automating the tuning of the hybrid interpolation coefficients to generalize across a wider variety of datasets and degradation types.
Summary updated