Introduction: The Fidelity-Editability Dilemma
To edit real-world images using diffusion models, practitioners must first "invert" the image—mapping it back into the model's latent noise space so it can be re-generated with new instructions. The standard tool for this, DDIM inversion, relies on a critical approximation: it assumes the noise predicted at one step is a perfect proxy for the noise at the next. This creates a small error that accumulates into a massive "trajectory drift."
This problem is severely amplified by Classifier-Free Guidance (CFG). Practitioners face a persistent "fidelity-editability dilemma": high guidance scales enable better instruction following but destroy the original image's details, while low scales preserve the image but ignore the edit instructions.
POLARIS: A Dynamic Approach to Inversion
POLARIS (Projection-Orthogonal Least Squares for Robust and Adaptive Inversion) introduces a robust, adaptive method for inverting diffusion models by treating the guidance scale as a dynamic variable rather than a fixed hyperparameter.
Instead of trying to compensate for errors after they happen (the approach taken by most "optimization-based" methods), POLARIS targets the origin of the error. By reframing the problem geometrically, the authors derived a closed-form formula for the optimal scale. The optimal scale is found by projecting the origin onto the line connecting the conditional and unconditional noise predictions. This yields a closed-form update rule that can be implemented in just one line of code, requiring no iterative optimization or backpropagation during the inversion process.
Key Results and Efficiency
The researchers evaluated POLARIS across several large-scale benchmarks, including MS-COCO 2017, Pick-a-Pic, and EditBench. The results demonstrate that fixing the foundation of the inversion path is more effective than adding corrective layers:
- Reconstruction Fidelity: POLARIS achieves massive gains in reconstruction. On COCO2017, it boosted PSNR from a baseline of ~14 dB to over 22 dB.
- Efficiency: Unlike optimization-based methods that can take minutes per image, POLARIS adds only ~3% computational overhead to the standard DDIM baseline. It is significantly faster than alternatives like EDICT or Null-Text Inversion.
- Downstream Versatility: The method showed significant fidelity improvements across colorization, inpainting, deblurring, and super-resolution. In editing tasks, it demonstrated a superior ability to preserve background details while successfully executing complex object replacements.
Conclusion and Future Work
POLARIS shifts the view of the guidance scale from a static "knob" to a dynamic "control law" that adapts to the specific geometry of the latent space at each timestep. While the method excels at object and attribute manipulation, complex non-rigid edits (such as significant pose changes) remain a challenge for current inversion-denoising paradigms. Future directions include stabilizing the "exact" mathematical solution for even higher fidelity and adapting the framework for video inversion and 3D generation tasks.
Summary updated