Wavelet-Guided Acceleration of Text Inversion in Diffusion-Based Image Editing

Gwanhyeong Koo, Sunjae Yoon, Chang D. Yoo

Abstract

Null-text Inversion (NTI) is a cornerstone for high-fidelity image editing in diffusion models, yet its computational cost remains a significant barrier to real-time application. This paper introduces the WaveOpt-Estimator, a novel wavelet-based framework designed to predict the optimal early-exit point for the optimization process. By leveraging the frequency characteristics of images, the proposed method reduces processing time by over 80% while maintaining reconstruction quality nearly indistinguishable from full-step NTI.

Introduction

Diffusion models have transformed the landscape of image generation and editing. However, editing real-world images requires an initial "inversion" step—mapping the image back to a noise latent that can accurately recreate the original during denoising. While Null-text Inversion (NTI) provides high-fidelity results by optimizing text embeddings at every timestep, it is computationally expensive, often requiring several minutes per image. Faster alternatives like Negative-Prompt Inversion (NPI) exist but frequently compromise structural integrity and fine details. This research addresses the efficiency-quality trade-off by identifying that exhaustive optimization is often redundant, particularly for images with specific frequency profiles.

Methodology: The WaveOpt-Estimator

The core innovation of this work is the WaveOpt-Estimator, which dynamically determines the optimal stopping point ($t^*$) for optimization. The approach is rooted in the observation that image convergence rates in the diffusion process are heavily influenced by their frequency components.

Frequency Analysis via Wavelets

Using Discrete Wavelet Transform (DWT), the authors decompose images into low-frequency ($x_{LL}$) and high-frequency ($x_{SUM}$) subbands. Empirical analysis revealed that images rich in high-frequency textures reach high-quality reconstruction thresholds significantly faster than those dominated by low-frequency global structures.

Architecture

The WaveOpt-Estimator utilizes a ResNet-50 backbone to process the original image, its wavelet subbands, and the current latent state ($z_t$). These features are integrated through cross-attention and self-attention modules to predict the specific timestep where further optimization yields diminishing returns.

The WaveOpt-Estimator architecture integrates image features and wavelet subbands to predict the optimal optimization endpoint.

Optimization Strategy

Instead of the standard 50-step optimization, the system optimizes only until the predicted $t^*$. Beyond this point, the last optimized embedding is reused. To further accelerate the process, the system initializes optimization using a target prompt (derived from NPI) rather than a generic null-text string.

Experimental Results

The model was evaluated on a dataset of 50 images commonly used in diffusion editing literature, generating a training set of 750 samples. Performance was measured using Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity Index (SSIM).

Speed and Quality Trade-offs

The proposed "NPI + WaveOpt-Estimator" method achieves a dramatic reduction in latency. While standard NTI requires approximately 180 seconds, the proposed method completes in just 28 seconds—an 84% speedup. Critically, this efficiency does not come at the cost of quality; the method maintains a PSNR ratio of 0.94 relative to the original, compared to only 0.82 for standard NPI.

Qualitative comparison of reconstruction and editing tasks across different inversion methods, highlighting the speed and fidelity of the proposed estimator.

Accuracy of Prediction

The estimator demonstrated high reliability, predicting the optimal stopping point with a Mean Absolute Error (MAE) of only 2.9 timesteps. This precision ensures that the "sweet spot" of reconstruction—where visual differences become negligible to the human eye—is consistently captured.

Conclusion and Future Work

The WaveOpt-Estimator successfully bridges the gap between slow, high-quality inversion and fast, low-quality alternatives. By making the inversion process frequency-aware, the authors provide a scalable path toward real-time diffusion-based editing. Future research will focus on expanding the training dataset to broader domains like COCO and exploring the compatibility of wavelet-guided optimization with emerging ultra-fast samplers such as Latent Consistency Models (LCM).

Summary updated

Abstract

In the field of image editing, Null-text Inversion (NTI) enables fine-grained editing while preserving the structure of the original image by optimizing null embeddings during the DDIM sampling process. However, the NTI process is time-consuming, taking more than two minutes per image. To address this, we introduce an innovative method that maintains the principles of the NTI while accelerating the image editing process. We propose the WaveOpt-Estimator, which determines the text optimization endpoint based on frequency characteristics. Utilizing wavelet transform analysis to identify the image’s frequency characteristics, we can limit text optimization to specific timesteps during the DDIM sampling process. By adopting the Negative-Prompt Inversion (NPI) concept, a target prompt representing the original image serves as the initial text value for optimization. This approach maintains performance comparable to NTI while reducing the average editing time by over 80% compared to the NTI method. Our method presents a promising approach for efficient, high-quality image editing based on diffusion models.
BibTeX
@inproceedings{koo2024wavelet,
  author = {Gwanhyeong Koo and Sunjae Yoon and Chang D. Yoo},
  title = {{Wavelet-Guided Acceleration of Text Inversion in Diffusion-Based Image Editing}},
  booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing},
  year = {2024},
  url = {https://openreview.net/forum?id=br40bs9Wjf},
}
Figure