POLARIS: Projection-Orthogonal Least Squares for Robust and Adaptive Inversion in Diffusion Models

Wenshuo Chen, Haosen Li, Shaofeng Liang, Lei Wang, Haozhe Jia, Kaishen Yuan, Jieming Wu, Bowen Tian, Yutao YuearXiv 2025 · DOI · arXiv

Introduction: The Fidelity-Editability Dilemma

To edit real-world images using diffusion models, practitioners must first "invert" the image—mapping it back into the model's latent noise space so it can be re-generated with new instructions. The standard tool for this, DDIM inversion, relies on a critical approximation: it assumes the noise predicted at one step is a perfect proxy for the noise at the next. This creates a small error that accumulates into a massive "trajectory drift."

This problem is severely amplified by Classifier-Free Guidance (CFG). Practitioners face a persistent "fidelity-editability dilemma": high guidance scales enable better instruction following but destroy the original image's details, while low scales preserve the image but ignore the edit instructions.

POLARIS: A Dynamic Approach to Inversion

POLARIS (Projection-Orthogonal Least Squares for Robust and Adaptive Inversion) introduces a robust, adaptive method for inverting diffusion models by treating the guidance scale as a dynamic variable rather than a fixed hyperparameter.

Instead of trying to compensate for errors after they happen (the approach taken by most "optimization-based" methods), POLARIS targets the origin of the error. By reframing the problem geometrically, the authors derived a closed-form formula for the optimal scale. The optimal scale is found by projecting the origin onto the line connecting the conditional and unconditional noise predictions. This yields a closed-form update rule that can be implemented in just one line of code, requiring no iterative optimization or backpropagation during the inversion process.

POLARIS demonstrates significant improvements across various tasks including colorization, inpainting, and super-resolution, while maintaining high reconstruction fidelity during the inversion process.

Key Results and Efficiency

The researchers evaluated POLARIS across several large-scale benchmarks, including MS-COCO 2017, Pick-a-Pic, and EditBench. The results demonstrate that fixing the foundation of the inversion path is more effective than adding corrective layers:

  • Reconstruction Fidelity: POLARIS achieves massive gains in reconstruction. On COCO2017, it boosted PSNR from a baseline of ~14 dB to over 22 dB.
  • Efficiency: Unlike optimization-based methods that can take minutes per image, POLARIS adds only ~3% computational overhead to the standard DDIM baseline. It is significantly faster than alternatives like EDICT or Null-Text Inversion.
  • Downstream Versatility: The method showed significant fidelity improvements across colorization, inpainting, deblurring, and super-resolution. In editing tasks, it demonstrated a superior ability to preserve background details while successfully executing complex object replacements.
A detailed look at image editing: POLARIS successfully executes the prompt 'Convert a cat to a dog' while preserving the complex background (the tennis racket), outperforming the baseline in both visual quality and PSNR.

Conclusion and Future Work

POLARIS shifts the view of the guidance scale from a static "knob" to a dynamic "control law" that adapts to the specific geometry of the latent space at each timestep. While the method excels at object and attribute manipulation, complex non-rigid edits (such as significant pose changes) remain a challenge for current inversion-denoising paradigms. Future directions include stabilizing the "exact" mathematical solution for even higher fidelity and adapting the framework for video inversion and 3D generation tasks.

Summary updated

Abstract

The Inversion-Denoising Paradigm, which is based on diffusion models, excels in diverse image editing and restoration tasks. We revisit its mechanism and reveal a critical, overlooked factor in reconstruction degradation: the approximate noise error. This error stems from approximating the noise at step t with the prediction at step t-1, resulting in severe error accumulation throughout the inversion process. We introduce Projection-Orthogonal Least Squares for Robust and Adaptive Inversion (POLARIS), which reformulates inversion from an error-compensation problem into an error-origin problem. Rather than optimizing embeddings or latent codes to offset accumulated drift, POLARIS treats the guidance scale ω as a step-wise variable and derives a mathematically grounded formula to minimize inversion error at each step. Remarkably, POLARIS improves inversion latent quality with just one line of code. With negligible performance overhead, it substantially mitigates noise approximation errors and consistently improves the accuracy of downstream tasks.
BibTeX
@misc{chen2025polaris,
  author = {Wenshuo Chen and Haosen Li and Shaofeng Liang and Lei Wang and Haozhe Jia and Kaishen Yuan and Jieming Wu and Bowen Tian and Yutao Yue},
  title = {{POLARIS: Projection-Orthogonal Least Squares for Robust and Adaptive Inversion in Diffusion Models}},
  year = {2025},
  eprint = {2512.00369},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2512.00369},
}
Figure