ConserveEdit: Source-Anchored Transactions for Drift-Bounded Iterative Image Editing

Shuyang Jiang, Fucheng Deng, Yuchuan Luo, Zhenyu WuarXiv 2026 · arXiv

TL;DR

ConserveEdit is an editor-agnostic controller that prevents "drift"—unintended changes to backgrounds or protected objects—during iterative image editing. By treating each refinement step as a transaction against a "contract" anchored to the original source image, it ensures that pixels outside the edit region remain bit-identical to the original while strictly auditing for edit progress and image quality.

What Problem Does This Paper Address?

Instruction-driven image editors (like InstructPix2Pix or Qwen-Image-Edit) are impressive, but they struggle with iterative refinement. Because these models typically re-render the entire scene for every edit, small unintended changes accumulate over multiple turns. This is known as drift.

Even when using masks, standard editors often cause global color shifts or distort nearby sensitive content like faces and text. Simple masking isn't a silver bullet because automatic grounding is imperfect; if a mask slightly overlaps a person's face, a "make the train smile" instruction might accidentally hallucinate features onto that face. The authors argue that current "agentic" editors fail because they rely on aggregate quality scores that allow a strong edit to "compensate" for a corrupted background.

Methods

ConserveEdit introduces a transactional refinement loop that treats the original source image ($x_0$) as the immutable source of truth.

  1. The Edit Contract: Before editing begins, a contract is compiled from the source image and the instruction. It defines the authorized support (the "core" mask of what should change plus a "halo" for shadows/reflections) and protected entities (background, faces, and text) with explicit tolerances.
  2. Source-Anchored Masking: Every candidate generated by a frozen editor is composed with the source image through a mask. This ensures that every pixel outside the authorized region is bit-exact to the original $x_0$.
  3. Multi-Dimensional Audit: Instead of one aggregate score, ConserveEdit subjects each candidate to three independent tests:
  4. Progress: Does the edit actually follow the instruction?
  5. Drift: Have protected entities (faces, text) changed beyond a calibrated threshold relative to $x_0$?
  6. Quality: Are there visible seams or artifacts at the mask boundaries?
  7. Rollback and Repair: If no candidate passes all tests, the system rolls back the transaction and attempts a "repair," such as shrinking the mask to avoid a protected entity or expanding the halo to capture a missing shadow.
Figure 2. ConserveEdit on a real MagicBrush test turn with a frozen Qwen-Image-Edit. 1 The contract is compiled once from x0; the region outside the authorized support A=M ∪H is protected. 2 Every raw candidate changes the whole image, so a scalar loop would have to trade that drift against progress. Each candidate is instead composed through the commit mask and audited for progress and quality as separate conditions. 3 The feasible candidate with the largest progress is committed and every decision is logged. Shown is the first step of the run; all images and numbers come from the released code (Sec. 4.1).

Datasets

  • MagicBrush: The primary real-world benchmark, consisting of 1,053 multi-turn image editing sequences with human-drawn masks and ground-truth targets.
  • SimEdit: A new procedural simulator introduced by the authors. It generates scenes with known ground-truth masks for targets, shadows, faces, and text, allowing for precise measurement of drift that is impossible with real-world data where the "true" causal support of an edit is unknown.

Key Insights and Ideas

  • Goalpost Drift: Most iterative editors compare the current step to the previous step. ConserveEdit always compares back to the original source. This prevents the "telephone game" effect where small errors compound into total image degradation.
  • Masking is Necessary but Not Sufficient: As shown in Figure 1, even with a mask, direct editing often fails to protect nearby sensitive content. Masking provides bit-exactness for the background, but the audit is what protects the content inside or near the mask.
  • Non-Substitutability: The authors prove that no single scalar weight (e.g., $EditScore - \lambda \cdot Drift$) can replicate their results. You cannot "buy" a better edit by sacrificing the integrity of a human face.
Figure 1. Iterating a strong editor accumulates drift; source-anchored transactions do not. Left: two MagicBrush test turns with a frozen Qwen-Image-Edit, outputs (top; blue: edit mask) and per-pixel change |x −x0| with the PSNR outside the dilated mask (bottom; changes below 0.15 from resampling the gallery renders are hidden). One direct call already re-renders the scene, and the naive loop zooms in and repaints it. ConserveEdit commits only masked candidates that pass separate progress, drift, and quality tests, so everything outside the authorized region A stays bit-identical to x0. Right: next to an automatic mask (Tab. 3), masking alone changes a protected face or text on 48% of turns, and masked scalar selection avoids this mostly by abstaining; the separate tests of ConserveEdit keep violations at 2% and raise constrained success (CS) to 53%. Turns: two of the 19 committed turns of the supplementary gallery, chosen as clear examples (their seams are typical, Sec. 5); Fig. 3 and the gallery are uncurated.

Key Results

  • Bit-Exact Preservation: ConserveEdit guarantees 0% drift for all pixels outside the authorized region.
  • Face/Text Protection: On MagicBrush turns involving automatic masks, standard masked editing violated protected faces or text 48% of the time. ConserveEdit reduced this to just 2%.
  • Constrained Success (CS): On the same task, ConserveEdit achieved a CS of 53%, more than double the 23% achieved by standard scalar selection methods.
  • DINO Similarity: Using a frozen Qwen-Image-Edit, ConserveEdit raised the DINO similarity to ground truth from 0.67 (direct) to 0.85, matching the performance of human-masked baselines but with automated protection.

Limitations and Future Work

  • Grounding Bottleneck: The system relies heavily on the initial grounding (e.g., Grounding DINO + SAM). If the initial mask misses the target entirely, the edit will fail or be incomplete.
  • Boundary Seams: While the quality audit helps, re-rendering editors often produce systematic seams at mask boundaries. Future work could integrate better harmonization or diffusion-based blending within the halo.
  • Predicate Noise: The local CLIP-based progress scores can be noisy, sometimes leading to "false rejections" where a valid edit is discarded because the score didn't hit a specific threshold.
  • Computational Cost: Generating multiple candidates and running multiple audits per turn is significantly more expensive than a single "direct" edit call.

Summary updated

Abstract

Instruction-driven image editors produce convincing local edits in one pass, but inside an iterative refinement loop every accepted output becomes the next input, so unintended changes outside the requested region accumulate. We present ConserveEdit, an editor-agnostic controller that treats each refinement step as a transaction against a contract compiled once from the source image. The contract separates what may change (the authorized region of each target, with a bounded halo for shadows and reflections) from what must survive (protected content with explicit tolerances). Candidates from a frozen editor are composed through a source-anchored mask and committed only if target progress, drift on protected content, and image quality pass as separate tests; otherwise the step is rolled back and repaired. For any editor, this keeps every pixel outside the authorized region bit-exact and bounds drift independently of the number of iterations, whereas no fixed scalar trade-off reproduces the separate tests. On all 1053 MagicBrush test edits with a frozen Qwen-Image-Edit, masking is necessary but not sufficient. A direct call already alters the rest of the image (12.1 dB PSNR outside the mask); composed through the source-anchored mask, it stays bit-exact there and its DINO similarity to the ground truth rises from 0.67 to 0.85, matching ConserveEdit, so this gain comes from the mask. Next to automatic masks, however, masked direct editing changes a protected face or text on 48% of turns, and masked scalar selection avoids this mostly by abstaining (23% constrained success). Because its drift test names the violated entity, ConserveEdit shrinks the support instead (2% violations, 53% constrained success); in a simulator with known ground truth it raises constrained success from 55.0% to 61.8%. Automatic grounding remains the main bottleneck. Code and logs will be released.
BibTeX
@misc{jiang2026conserveedit,
  author = {Shuyang Jiang and Fucheng Deng and Yuchuan Luo and Zhenyu Wu},
  title = {{ConserveEdit: Source-Anchored Transactions for Drift-Bounded Iterative Image Editing}},
  year = {2026},
  eprint = {2611.22724},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.22724},
}
Figure