Test-Time Preference Optimization for Image Restoration

Bingchen Li, Xin Li, Jiaqi Xu, Jiaming Guo, Wenbo Li, Renjing Pei, Zhibo Chen

Abstract

Image restoration (IR) models traditionally optimize for pixel-wise metrics like PSNR, often leading to over-smoothed results that lack realistic textures. We present Test-Time Preference Optimization (TTPO), a training-free framework that enhances IR outputs to better align with human aesthetic preferences. TTPO utilizes pre-trained diffusion models to generate candidate enhancements at test-time, ranks them using automated quality metrics, and optimizes the final output through a conditioned denoising process. Our approach preserves structural integrity while significantly improving perceptual quality across various IR tasks, including denoising, super-resolution, and deraining.

Introduction

Standard image restoration (IR) techniques—such as denoising, deblurring, and super-resolution—typically rely on training objectives like L1 or Mean Squared Error (MSE). While these objectives yield high Peak Signal-to-Noise Ratio (PSNR) scores, they frequently produce "plastic" or blurry images that fail to satisfy human aesthetic judgment. In contrast, the field of Text-to-Image (T2I) generation has successfully employed Reinforcement Learning from Human Feedback (RLHF) to align model outputs with human tastes.

Applying RLHF to IR, however, presents unique challenges. First, there is a significant lack of large-scale preference datasets specifically for restoration. Second, IR models are architecturally diverse and often deterministic, making them difficult to integrate into standard RL policy frameworks. Finally, IR requires strict adherence to the structural layout of the input image, whereas T2I allows for more creative deviations. To address these issues, we propose Test-Time Preference Optimization (TTPO), a post-processing framework that acts as a "booster" for any existing IR backbone.

Methodology

TTPO operates through a three-stage pipeline executed at test-time, requiring no retraining of the base IR model.

The three-stage TTPO pipeline: Generation, Selection, and Optimization.

1. Generation: Online Preference Data

Starting with an initially restored image ($y_0$), the system employs diffusion editing (using models like SD3 or FLUX) to generate multiple variations. By introducing a controlled amount of noise and then denoising, the diffusion model "hallucinates" realistic textures and fine details based on its extensive pre-trained priors.

2. Selection: Automated Feedback

To circumvent the need for slow human labeling during inference, TTPO uses a hybrid of three No-Reference Image Quality Assessment (NR-IQA) metrics: MUSIQ, MANIQA, and Q-Align. These metrics rank the generated candidates. The highest-scoring image is designated as the "preferred" ($y_w$) sample, while the lowest-scoring one is the "dispreferred" ($y_l$) sample.

3. Optimization: Conditioned Denoising

The initial image $y_0$ is refined through a conditioned diffusion process. To ensure structural fidelity while improving texture, TTPO utilizes frequency decomposition:

  • High Frequencies: These are optimized using a preference loss ($L_{TTPO}$) that steers the image toward the textures of $y_w$ and away from $y_l$.
  • Low Frequencies: These are constrained by a structural consistency loss ($L_r$) to ensure the global layout remains identical to the original input.

Experiments and Results

We evaluated TTPO across a comprehensive suite of IR benchmarks, including Kodak24, RealSR, and Rain100L, covering tasks such as denoising, super-resolution, and deraining.

Quantitative and Qualitative Performance

TTPO consistently improved perceptual scores across 12 different quantitative comparisons. For instance, in denoising tasks using SwinIR, TTPO raised the MANIQA score from 0.4565 to 0.4912. Qualitatively, the results demonstrate a marked improvement in the realism of hair, skin textures, and architectural details.

Qualitative comparison of the initial restoration ($y_0$) vs. the TTPO-optimized result ($y_{TTPO}$).

Key Insights

A significant finding is that the optimized image ($y_{TTPO}$) often outperforms the "best" candidate ($y_w$) used for guidance. This suggests that the optimization process successfully synthesizes the structural accuracy of the original IR output with the superior aesthetic textures of the diffusion-generated samples. Furthermore, the method is highly flexible, proving effective regardless of whether the underlying IR model is based on CNNs, Transformers, or zero-shot diffusion methods. In a user study, human experts preferred TTPO-optimized images over baseline outputs more than 70% of the time.

Discussion and Conclusion

While TTPO offers a powerful way to enhance IR quality, it faces limitations regarding computational overhead. Running high-end diffusion models like FLUX at test-time currently takes between 26 and 45 seconds per image, which is not yet suitable for real-time applications. Additionally, while NR-IQA metrics are effective, they do not perfectly mirror human preference in all edge cases.

Future work will focus on developing more efficient diffusion sampling techniques and refining automated preference selection mechanisms to further bridge the gap between algorithmic metrics and human perception. Despite these challenges, TTPO demonstrates that large-scale diffusion models can serve as potent aesthetic priors for image restoration without the need for task-specific fine-tuning.

Summary updated

BibTeX
@inproceedings{li2026test,
  author = {Bingchen Li and Xin Li and Jiaqi Xu and Jiaming Guo and Wenbo Li and Renjing Pei and Zhibo Chen},
  title = {{Test-Time Preference Optimization for Image Restoration}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2026},
  doi = {10.1609/aaai.v40i8.37520},
}
Figure