Abstract
LASPA (LAtent SPatial Alignment) is a novel, training-free framework designed for high-quality text-based editing of real images. By aligning the spatial latents of a diffusion model with the reference image's features during the denoising process, LASPA achieves state-of-the-art results in under 6 seconds. Unlike existing methods, it requires no model finetuning or complex optimization, making it highly efficient for both computation and storage.
Introduction
Text-to-image diffusion models have revolutionized content creation, yet editing existing real images remains computationally expensive. Current state-of-the-art techniques typically rely on inverting images into the latent space or finetuning model parameters to capture specific object identities. These processes are often slow, taking several minutes per image, and require significant storage for per-image model weights.
LASPA addresses these challenges by introducing an inference-only guidance mechanism. It preserves the original image's identity and structure while faithfully following new textual instructions, all without modifying the underlying model weights.
Methodology: Latent Spatial Alignment
The core of the LASPA framework is the guidance of spatial latents throughout the reverse diffusion process. The authors argue that spatial latents are inherently superior to textual tokens for encoding fine-grained visual details, bypassing the "information bottleneck" often found in text-based inversion.
Alignment Strategies
The researchers evaluated three primary alignment methods: Input Alignment, epsilon-theta Alignment, and x0 Prediction Alignment. The study concluded that aligning the x0 prediction (Equation 9 in the paper) offers the optimal balance. Early in the diffusion process, the model is steered toward the original image's structure. As denoising progresses, this alignment is gradually relaxed, providing the model with the "freedom" to incorporate textual edits.
Semantic Latent Mixing
To further enhance precision, LASPA utilizes Semantic Latent Mixing. This technique leverages the model's internal attention maps as masks, allowing for localized edits (e.g., background replacement) while keeping the primary subject unchanged. Notably, LASPA generates these clean attention maps in a single forward pass, whereas competing methods often require extensive optimization.
Results and Performance
LASPA was evaluated using the TEdBench benchmark and various base models, including Stable Diffusion (v1.4, v2.1), Wuerstchen, and Latent Consistency Models (LCMs).
Speed and Efficiency
LASPA performs edits in less than 6 seconds, a significant improvement over methods like Imagic (~8 minutes) and SINE (~15 minutes). Furthermore, it requires 0GB of additional storage, as it operates on frozen, pre-trained weights.
Quality and User Preference
In a user study involving over 1,200 responses, LASPA was preferred 71% of the time over SDEdit and 62% of the time over SINE. Quantitative metrics (LPIPS for preservation and CLIP similarity for editing strength) further validate that LASPA maintains high image fidelity without sacrificing the quality of the requested edits.
Conclusion
LASPA provides a fast, training-free solution for single-image editing. While it excels at maintaining structure and style, it faces challenges with large-scale geometric transformations (e.g., significant pose changes). Future research will explore combining spatial alignment with advanced control mechanisms to handle complex structural overhauls while maintaining the framework's efficiency.
Summary updated