Spectral Evolution Search: Efficient Inference-Time Scaling for Reward-Aligned Image Generation

Jinyan Ye, Zhongjie Duan, Zhiwen Li, Cen Chen, Daoyuan Chen, Yaliang Li, Yingda ChenICML 2026 · OpenReview

Abstract

Inference-time scaling offers a flexible way to align visual generative models with downstream objectives without updating model parameters. In modern image generation models, a natural way to do this is to optimize the random noise from which generation starts. However, searching in this high-dimensional noise space is highly inefficient, because many directions have little effect on the final image. We trace this inefficiency to a spectral bias in generative dynamics: model sensitivity to initial perturbations decays rapidly as frequency increases. Based on this insight, we propose Spectral Evolution Search (SES), a plug-and-play framework for initial noise optimization that performs gradient-free evolutionary search in a low-frequency subspace. Theoretically, we derive the Spectral Scaling Prediction from perturbation propagation dynamics, which explains the frequency-dependent impact of perturbations. Extensive experiments across diverse settings show that SES substantially improves the trade-off between generation quality and computational cost, consistently outperforming strong baselines under the same compute budget.
BibTeX
@inproceedings{ye2026spectral,
  author = {Jinyan Ye and Zhongjie Duan and Zhiwen Li and Cen Chen and Daoyuan Chen and Yaliang Li and Yingda Chen},
  title = {{Spectral Evolution Search: Efficient Inference-Time Scaling for Reward-Aligned Image Generation}},
  booktitle = {International Conference on Machine Learning},
  year = {2026},
  url = {https://openreview.net/forum?id=tBt12b9dxa},
}

Loading related content...

Figure