Editing a real photo with a text-to-image diffusion model (a generator that turns random noise into an image by removing noise step by step) starts with inversion: running the denoising process backwards to find the noise that would regenerate the photo. That recovered noise is not really random. It carries a faint "ghost" of the original image's outlines. When you ask for a change of shape, such as a butterfly becoming a parrot, the ghost pulls the edit back toward the old silhouette. The usual fixes make one of two trades. They either keep the ghost and get faithful but rigid edits, or they scramble it with fresh noise and get flexible edits that no longer match the source. We propose that this trade-off is mostly a matter of which spatial frequencies are constrained, where in the image, and at which denoising steps. The outline that blocks a shape change lives in the coarse, low-frequency part of the image, inside the region being edited, and it matters at the early high-noise steps. The textures and background that make a reconstruction look faithful live in finer bands, or outside that region, or at later steps.
The method is small. After a standard inversion, we decompose the latent with a wavelet transform, which splits a signal into coarse and fine sub-bands while keeping track of where each coefficient sits in the image. We then do two things. First, we re-randomize only the coarse wavelet coefficients of the starting noise, and only inside the edited region. Second, we filter the source-preservation correction that dual-branch editors inject, so that it is withheld from the coarse bands of the edited region during early steps and applied everywhere else. If the ghost really is concentrated in the coarse bands, this should allow non-rigid edits at close to the background fidelity of the best reconstruction-oriented inversions. We test that premise directly before building on it.
Core Claim and What Is New
The claim is that a band-by-time-by-region gate on source preservation reaches a better balance than any of three simpler knobs: a single timestep knob (partial inversion, or replacing early inversion steps with forward noise), a single spatial knob (mask blending), or a single band knob that is fixed for all steps (feature-level frequency splitting). Each prior method moves along one of these axes. We claim the useful moves need all three together.
The evidence that motivates this comes from several papers. There and Back Again shows that DDIM-inverted latents are non-Gaussian and correlated with the source, especially in plain regions. Spectral Collapse in Diffusion Inversion shows that the inverted noise keeps a low-frequency imprint of the input while losing high-frequency energy. Editable Noise Map Inversion documents the concrete failure: non-rigid edits keep "original outlines and wing shapes." Taken together, these point to the coarse bands as the carrier of rigidity. None of these papers, however, measures ghost strength per band and per timestep on natural-image editing benchmarks. That measurement is our first deliverable.
The closest existing work falls into three groups, and our method differs from each.
The first group applies frequency ideas to inversion itself. WINE (earlier preprint WaGI) uses a multi-level wavelet loss and wavelet-domain fusion for GAN inversion. It depends on a wavelet-native generator (SWAGAN) and does not address diffusion trajectories. Spectral Collapse proposes Orthogonal Variance Guidance, which restores high-frequency variance while projecting it away from the structural gradient. That projection forbids exactly the structural change a non-rigid edit needs, a limitation the research file notes. Our gate does the opposite: it deliberately releases structure, but only locally and only in coarse bands.
The second group applies frequency ideas at sampling time. W-EDIT injects wavelet-split source features into selected transformer blocks. FIA-Edit fuses Fourier-split features inside attention and skips inversion entirely. Both modify features during sampling and leave the inverted noise and its correction untouched. FIA-Edit also explicitly reports difficulty with drastic shape changes.
The third group consists of dual-branch and optimization inversions: PnP Inversion, Editable Noise Map Inversion, and DCI. All of them correct the trajectory with spatial, all-band residuals or losses. These methods supply our machinery, and we reuse PnP's source-branch correction unchanged except for the gate.
Research Program
We work first in Stable Diffusion v1.5. PnP, ENM and DCI all report PIE-Bench results in this model. We then port the method to FLUX so we can compare against W-EDIT.
Step 1: diagnose the ghost. For DDIM inversion on PIE-Bench images, we decompose each inverted latent $z_t$ into Haar wavelet bands over three levels. The 64×64 latent supports about three levels. For each band and timestep we compute two quantities. The first is the correlation between that band of $z_t$ and the same band of the encoded source. The second is its deviation from the energy expected for Gaussian noise. We also ask whether ghost strength in a band predicts edit failure, using PIE-Bench's non-rigid categories (pose alteration, object replacement with a shape change). The measure of failure is the overlap between source edges and edited edges inside the annotated edit mask.
This step can falsify the idea. If ghost correlation is spread evenly across bands, or if it does not predict failure, band gating has no purchase and we stop.
Step 2: the gate. Let $r_t$ be the correction PnP adds to the target branch at step $t$: the difference between the inversion latent and the source-branch latent. Let $W_b$ extract wavelet band $b$, and let $m$ be a soft edit-region map. We replace the full correction with a gated one:
$$\tilde r_t = \sum_b W_b^{-1}\big( g_b(t, m) \odot W_b r_t \big)$$
In this expression, $g_b(t,m)\in[0,1]$ is set to $1-m$ for coarse bands when $t>\tau$, and to $1$ otherwise. Here $\tau$ is the step after which structure is re-bound. Everywhere else in the image, and in fine bands at all times, the correction passes through untouched. We apply the same mask and band set to the starting noise $z_T$. There, the coarse coefficients inside $m$ are resampled from a Gaussian with the band energy that pure noise would have.
The map $m$ is computed without annotations. We threshold the cross-attention maps of the tokens that differ between the source and target prompts. The PIE-Bench masks are used only for scoring.
Step 3: schedule. We treat $\tau$ and the number of released levels as the only hyperparameters. We sweep them on a held-out slice of PIE-Bench. We also test whether ENM's observation that edit success is decided around step 30 gives a good default for $\tau$ without tuning.
The added cost is a few wavelet transforms per step. Runtime should therefore stay close to PnP's reported ~28 seconds per image, and we measure it.
Baselines
The naive reference point is plain DDIM inversion followed by prompt-conditioned sampling. It shows how much rigidity and drift an ungated pipeline has.
The most direct competitor is PnP Inversion, which adds the full source-branch residual. It reports up to 83.2% better structure preservation (Structure Distance) than baseline inversions, at ~28 seconds per image. Our method is PnP with one change, so any gain is attributable to the gate. We run PnP with its reported best guidance (inverse scale 2.5, forward scale 7.5) on the same seeds and model.
The strongest reconstruction-oriented method is Editable Noise Map Inversion. It reports background SSIM of 86.29 and PSNR of 28.19 on PIE-Bench. DCI reports a reconstruction PSNR of 29.38 at 12.1 seconds per image. Of the prior methods, these two are the ones that also target editability through inversion. We rerun both from their released settings on our hardware rather than copying numbers, because the file does not report non-rigid-subset results for either.
For frequency-aware sampling on FLUX, the baseline is W-EDIT. It reports PSNR 24.06, CLIP score 31.84 and FID 65.44 on PIE-Bench. We also include FIA-Edit as the inversion-free frequency method. The file gives no single PIE-Bench number for FIA-Edit, only a best average rank. For Spectral Collapse / OVG there is no reported PIE-Bench result on Lacuna, so we implement it on top of DDIM as described.
Two ablations isolate the key ingredient. The first gives PnP the same attention-derived mask with an all-band spatial gate. This separates the frequency effect from the masking effect, and we regard it as the single most important comparison. The second applies the band gate globally, with no mask, to test whether spatial locality is needed.
Every method receives the same tuning budget: a sweep of at most 16 configurations of its main knob on the same held-out slice.
We count a result as beating a baseline only under two conditions. First, on the non-rigid categories, our method must improve target CLIP score and reduce source-edge overlap inside the mask. Second, it must do so while keeping background PSNR and SSIM outside the mask within one standard error of ENM. Winning on only one of these axes does not count.
Why not just…?
Why not just replace the first inversion steps with forward noise? There and Back Again shows that replacing about 4% of steps restores Gaussianity. Replacing more causes hallucination, and Spectral Collapse reports structural drift from naive noise injection. That approach decorrelates every band everywhere, which is precisely the damage our gate avoids. We sweep the replaced fraction and plot its full trade-off curve against ours. If that curve matches ours on the non-rigid subset, the frequency gate adds nothing, and we will report that.
Why not just start editing from an intermediate timestep, or use a mask with PnP? These are the real threats, because both are one-knob controls that practitioners already use. Partial inversion is a single timestep knob, and masked PnP is a single spatial knob. Our claim is specifically that the band axis adds something beyond both. The masked all-band ablation described above, plus a partial-inversion sweep, will settle this question directly.
Why not a Gaussian or Laplacian pyramid instead of wavelets? A Laplacian pyramid also gives spatially local, coarse-versus-fine bands. It may work just as well. We do not argue otherwise, and the file's claim that spatial filtering suffers low-frequency dominance applies to losses, not to gates. We run the gate with a Laplacian pyramid and with a global Fourier split. The Fourier split has no spatial localization, so it is expected to lose to both local decompositions. If the pyramid ties with wavelets, we report the contribution as band-local gating rather than as anything specific to wavelets.
Decisive Checks
We evaluate on all 700 PIE-Bench images, reporting results for each of the ten categories separately. For the diagnostic we add COCO images to cover cluttered scenes. We also report CelebA-HQ separately, because There and Back Again found that decorrelation matters less on faces with uniform backgrounds.
The metrics are background PSNR, SSIM and LPIPS outside the mask; whole-image and in-mask CLIP score; source-edge overlap inside the mask; and runtime.
The idea stands or falls on three outcomes. First, the Step 1 diagnostic must show ghost correlation concentrated in coarse bands and predictive of failure. Second, the method must beat masked all-band PnP on the non-rigid categories without losing background fidelity. Third, on rigid edits such as color and material changes, the gate must not cost anything relative to PnP. A null result on any of the first two tells us the rigidity problem is not spectral, which the field would also benefit from knowing.
References
- There and Back Again: On the relation between Noise and Image Inversions in Diffusion Models
- Spectral Collapse in Diffusion Inversion
- Editable Noise Map Inversion: Encoding Target-image into Noise For High-Fidelity Image Manipulation
- WINE: Wavelet-Guided GAN Inversion and Editing for High-Fidelity Refinement
- WaGI : Wavelet-based GAN Inversion for Preserving High-frequency Image Details
- W-EDIT: A Wavelet-Based Frequency-Aware Framework for Text-Driven Image Editing
- FIA-Edit: Frequency-Interactive Attention for Efficient and High-Fidelity Inversion-Free Text-Guided Image Editing
- PnP Inversion: Boosting Diffusion-based Editing with 3 Lines of Code
- DCI: Dual-Conditional Inversion for Boosting Diffusion-Based Image Editing