Spectral Evolution Search: Efficient Inference-Time Scaling for Reward-Aligned Image Generation
Lists
Only you see this
Abstract
Inference-time scaling offers a flexible way to align visual generative models with downstream objectives without updating model parameters. In modern image generation models, a natural way to do this is to optimize the random noise from which generation starts. However, searching in this high-dimensional noise space is highly inefficient, because many directions have little effect on the final image. We trace this inefficiency to a spectral bias in generative dynamics: model sensitivity to initial perturbations decays rapidly as frequency increases. Based on this insight, we propose Spectral Evolution Search (SES), a plug-and-play framework for initial noise optimization that performs gradient-free evolutionary search in a low-frequency subspace. Theoretically, we derive the Spectral Scaling Prediction from perturbation propagation dynamics, which explains the frequency-dependent impact of perturbations. Extensive experiments across diverse settings show that SES substantially improves the trade-off between generation quality and computational cost, consistently outperforming strong baselines under the same compute budget.
BibTeX
@inproceedings{ye2026spectral,
author = {Jinyan Ye and Zhongjie Duan and Zhiwen Li and Cen Chen and Daoyuan Chen and Yaliang Li and Yingda Chen},
title = {{Spectral Evolution Search: Efficient Inference-Time Scaling for Reward-Aligned Image Generation}},
booktitle = {International Conference on Machine Learning},
year = {2026},
url = {https://openreview.net/forum?id=tBt12b9dxa},
}Loading related content...