TRACE: Attribute Execution for Sequential Image Editing

Xiyang Zhang, Yuanhe Tian, Hongzhi Wang, Yan SongarXiv 2026 · arXiv

TL;DR

TRACE is a novel framework for sequential image editing that prevents "instruction drift" by separating desired attributes (like color) from their visual carriers (the object's surface). By tracking active goals and executing local appearance operations, it ensures that new edits—such as changing a cup's material to metal—don't accidentally undo previous successes, like a prior request to make that cup orange.

What Problem Does This Paper Address?

Sequential image editing allows users to build a result through a conversation (e.g., "Make the cup orange," then "Change its material to metal"). However, current AI editors often suffer from a "forgetting" problem. Because an image is a flat collection of pixels, a model might successfully change a cup's material to metal but, in doing so, revert its color back to the original white, effectively "undoing" the first turn.

The core challenge is that attributes like color and material occupy the same pixels. Standard methods that try to "remember" by retrieving previous images often fail because copying pixels from an old image restores the old color but also restores the old, obsolete material. TRACE addresses this by treating attributes as independent operations that can be reapplied to new surfaces as the edit progresses.

Methods

TRACE functions as an "attribute executor" wrapped around a frozen image editor (like FLUX.2). It consists of three main stages:

  1. Active-Goal Binding: A text-only interpreter tracks the "state" of the conversation. It identifies which requirements are still "live" (e.g., the cup should be orange) and which have been superseded by new instructions.
  2. Local Attribute Execution: This is the heart of the system. Instead of letting the diffusion model regenerate everything, TRACE uses a parametric color operator in the Oklab color space. It calculates the color "displacement" needed to reach the target color from a historical reference and applies that displacement to the current object's pixels.
  3. If the current object lacks the requested material (e.g., it's still plastic when it should be metal), TRACE generates a corrective image to provide a new "carrier" surface while preserving the rest of the scene.
  4. Complete-Request Selection: A Vision-Language Model (VLM) acts as a "checker." It evaluates candidate images to ensure they satisfy the current request and all previous active goals. It also performs tonal calibration to ensure color changes don't "wash out" natural highlights and shadows.
Figure 1. TRACE separates the desired attribute from the surface that carries it. Active goals identify what should persist. History supplies color parameters, while current or acquired pixels supply the surface on which they are executed. Complete-request checks select a candidate and verify its tonal calibration. Each attempt uses at most one corrective image for missing color evidence or a missing surface; supported material repair can repeat once when an attribute remains unconfirmed. Local candidates have priority over fully checked whole-image fallbacks. Images in this overview are illustrative.

Datasets

The authors evaluated TRACE using several benchmarks:

  • EdiVal: A benchmark for multi-turn editing consisting of 572 sequences.
  • MagicBrush: A large-scale dataset of real-world image editing instructions (1,053 independent and 535 multi-turn sessions).
  • COCO120: A newly curated set of 120 photographs used to test how well the system transfers to diverse, real-world objects and scenes.

Key Insights and Ideas

  • Attribute vs. Carrier: The most important "aha" moment is that an attribute (the "what") and its carrier (the "where") must be bound separately. A mask tells you where to edit, but it doesn't tell you how to reconcile a new material with an old color.
  • The Power of Local Execution: By applying edits as local operations rather than global regenerations, TRACE significantly improves scene preservation. It avoids the "background drift" common in sequential editing where the environment subtly changes with every turn.
  • Target-Blind Reading: To verify if a material change was successful without being biased by the object's color, the system uses a "target-blind" observer—a VLM that describes the object's material while the specific color goal is withheld from the prompt.
Figure 2. A case study: The vase (top) requires color recovery on an existing surface, while the cup (bottom) requires acquiring a new metallic surface (carrier) before applying the orange color.

Key Results

  • Joint Recovery: In a controlled experiment on color-material conversations, TRACE achieved a joint success rate (keeping both attributes correct) of 96/108, vastly outperforming the raw editor's 62/108.
  • Benchmark Performance: On the EdiVal benchmark, TRACE achieved the highest final instruction-following score (31.12%) and the best overall consistency score among compared controllers.
  • Qualitative Superiority: As shown in Figure 3, TRACE's tonal calibration preserves fine details (like the dark markings on a bird) that are often lost or "tinted" by standard color-transfer methods.
Figure 3. Tonal calibration ensures that dark markings are preserved during a color change, whereas simple affine execution might wash them out.

Limitations and Future Work

  • Scope of Attributes: The current implementation focuses heavily on color and material. The authors suggest that extending this "target-carrier separation" to more complex attributes like shape, pose, and lighting is a promising future direction.
  • VLM Dependency: The system's success is partially tied to the accuracy of the VLM "checker." If the VLM fails to perceive a subtle attribute, the system may reject a valid edit.
  • Computational Cost: Running multiple VLM checks and potentially multiple diffusion draws per turn increases the inference time compared to a simple single-pass editor.

Summary updated

Abstract

Given an image and successive user instructions, sequential image editing should return an updated image that satisfies each new request and preserves earlier requirements that remain active. This lets users develop a result without repeatedly restating or repairing their decisions. The difficulty is that a retrieved image can contain a valid color on an obsolete surface, while the current image may lack the newly requested material. Neither reference selection nor a precise object mask alone specifies how to reconcile attributes occupying the same region. We present TRACE, which separates the continuing attribute from its visual carrier and prioritizes local appearance operations under complete-request checks. Active-goal binding identifies what should persist, carrier selection supplies suitable current or newly generated pixels, and tonal calibration reduces tinting near highlights and shadows. Experiments across frozen editors, independent visual readers, new source images, and complete sequential benchmarks support joint color-material recovery and show how local execution preserves the surrounding scene.
BibTeX
@misc{zhang2026trace,
  author = {Xiyang Zhang and Yuanhe Tian and Hongzhi Wang and Yan Song},
  title = {{TRACE: Attribute Execution for Sequential Image Editing}},
  year = {2026},
  eprint = {2611.25648},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.25648},
}
Figure