Test-Time Preference Optimization for Image Restoration

Bingchen Li, Xin Li, Jiaqi Xu, Jiaming Guo, Wenbo Li, Renjing Pei, Zhibo Chen

Abstract

Image restoration (IR) models typically optimize for pixel-wise metrics like L1 or MSE, which often fail to align with human perceptual preferences, resulting in over-smoothed outputs. We introduce Test-Time Preference Optimization (TTPO), a training-free paradigm that enhances the perceptual quality of any IR model at inference time. TTPO generates candidate variations on-the-fly, selects preferred samples using automated metrics, and uses these as reward signals to guide a diffusion-based refinement process. Our approach is backbone-agnostic and requires no pre-existing preference datasets. Extensive experiments demonstrate that TTPO significantly improves visual fidelity and aligns more closely with human judgment than standard restoration baselines.

Introduction

Traditional Image Restoration (IR) tasks—such as denoising, super-resolution, and deraining—have long relied on deterministic models optimized for Signal-to-Noise Ratio (PSNR). While effective at reducing noise, these models often produce "plasticky" or blurry textures that lack the fine details humans expect. While Reinforcement Learning from Human Feedback (RLHF) has successfully aligned generative models in other domains, its application to IR is hindered by the deterministic nature of most IR architectures and a severe lack of large-scale preference datasets.

TTPO addresses these challenges by shifting preference optimization to the test-time phase. By acting as a flexible enhancement layer, it allows users to leverage the generative power of large-scale diffusion models to refine the outputs of existing IR backbones without requiring expensive retraining.

The TTPO Paradigm

The three-stage TTPO pipeline: Generation of candidates, Selection via metrics, and Conditioned Optimization.

The TTPO framework operates through a three-stage, training-free pipeline:

  1. Candidate Generation: Starting with an initially restored image ($y_0$) from a base model (e.g., SwinIR), the system performs diffusion inversion to add controlled noise. It then generates multiple diverse candidate images by denoising this latent using different scales and pre-trained diffusion models like SD3 or FLUX.
  2. Automated Selection: To eliminate the need for real-time human intervention, TTPO employs a "committee" of No-Reference Image Quality Assessment (NR-IQA) metrics—MUSIQ, MANIQA, and Q-Align. These metrics rank the candidates to identify a "win" sample ($y_w$) representing high perceptual quality and a "lose" sample ($y_l$) representing lower quality or artifacts.
  3. Conditioned Optimization: The initial image $y_0$ is refined via a conditioned diffusion process. At each step, a gradient-based reward function nudges the image toward the features of $y_w$ while steering it away from $y_l$. To prevent "hallucinations" and maintain structural integrity, the optimization is applied primarily to high-frequency components (textures), while low-frequency components (global structure) are strictly constrained to match $y_0$.

Experimental Evaluation

We evaluated TTPO across diverse benchmarks, including Kodak24 (denoising), RealSR (super-resolution), and Rain100L (deraining). The method consistently outperformed base models across perceptual metrics such as MUSIQ and LIQE. For instance, in denoising tasks using SwinIR, TTPO improved the MUSIQ score from 73.79 to 74.24.

Human preference win rates for TTPO-optimized images against various baselines.

Crucially, human preference studies confirmed the effectiveness of the approach. In expert evaluations, images optimized by TTPO were preferred over the base model's output 70.31% of the time. Qualitative analysis shows that TTPO successfully restores realistic fine details, such as hair textures and sharp edges, which are often lost in standard restoration processes.

Conclusion

Test-Time Preference Optimization offers a powerful, backbone-agnostic solution for aligning image restoration with human visual preferences. By leveraging diffusion models as test-time refiners and using automated metrics for selection, TTPO bypasses the need for massive preference datasets and model retraining. While the current implementation faces computational overhead due to iterative diffusion steps, it sets a new precedent for flexible, high-fidelity image restoration.

Summary updated

Abstract

Image restoration (IR) models are typically trained to recover high-quality images using L1 or LPIPS loss. To handle diverse unknown degradations, zero-shot IR methods have also been introduced. However, existing pre-trained and zero-shot IR approaches often fail to align with human preferences, resulting in restored images that may not be favored. This highlights the critical need to enhance restoration quality and adapt flexibly to various image restoration tasks or backbones without requiring model retraining and ideally without labor-intensive preference data collection. In this paper, we propose the first Test-Time Preference Optimization (TTPO) paradigm for image restoration, which enhances perceptual quality, generates preference data on-the-fly, and is compatible with any IR model backbone. Specifically, we design a training-free, three-stage pipeline: (i) generate candidate preference images online using diffusion inversion and denoising based on the initially restored image; (ii) select preferred and dispreferred images using automated preference-aligned metrics or human feedback; and (iii) use the selected preference images as reward signals to guide the diffusion denoising process, optimizing the restored image to better align with human preferences. Extensive experiments across various image restoration tasks and models demonstrate the effectiveness and flexibility of the proposed pipeline.
BibTeX
@inproceedings{li2026test,
  author = {Bingchen Li and Xin Li and Jiaqi Xu and Jiaming Guo and Wenbo Li and Renjing Pei and Zhibo Chen},
  title = {{Test-Time Preference Optimization for Image Restoration}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2026},
  doi = {10.1609/aaai.v40i8.37520},
}
Figure