Logical Dependency Masks for Local Image Editing

When you ask an image editor to remove a cup from a table, two things should happen. The cup should disappear, and so should its shadow, its reflection in the window, and the dent it left in the napkin. Nothing else should change. Current editors fail in one of two ways. Some redraw too much: the lighting, textures, or people elsewhere in the picture drift, and the drift compounds over several rounds of edits. Others lock everything outside a user-drawn mask, which leaves "ghost shadows" and seams behind. The underlying problem is that no editor represents which parts of the background depend on the edited object. We propose to compute that dependency explicitly. A small set of logical rules (for example, "a shadow depends on the object casting it" or "an object resting on a support moves with the support") is run over a scene graph extracted from the image. The output divides the image into three zones: pixels that must change, pixels that may change, and pixels that must not change. Any mask-conditioned editor can then use these zones in place of a single hand-drawn mask.

The approach borrows two ideas from tractable reasoning over ontologies, where large logical knowledge bases are rewritten into a restricted rule language that can be evaluated in polynomial time. The first is that compound queries can be broken into canonical templates and evaluated cheaply. The second is that a problem too hard to solve exactly can be bracketed between a lower bound with no false positives and an upper bound with no misses. We apply these ideas to edits rather than to ontologies. The new claim is that background preservation becomes a two-sided, checkable target: the "must-preserve" zone is a sound lower bound on what may stay untouched, and the "may-change" zone is a complete upper bound on what the edit should be allowed to touch. Both are derived by deterministic rule evaluation, not left to a black-box model.

Core Claim

Leakage, meaning unintended changes to unedited regions, and ghost artifacts, meaning dependent regions left unchanged, are two errors of the same mask. They come from the same missing quantity: the set of image regions logically dependent on the edit. Suppose that set is computed by forward-chaining Horn rules (if-then rules with a single conclusion, the class of rules whose evaluation is polynomial in data size) over a perceived scene graph. Then, with the editing model held fixed, both kinds of error can be reduced at once. Nearer-to-hand fixes such as mask dilation or regularization strength only trade one error against the other.

The bracketing comes from perceptual uncertainty, not from logical intractability. Relation facts the perception module is confident about, such as a detected cast shadow or a detected support contact, enter a "certain" fact base. Low-confidence relations enter a "possible" fact base. The rules compute a set of affected entities from each base:

  • $A^{-}$ is the affected set derived from the certain facts only.
  • $A^{+}$ is the affected set derived from the certain plus possible facts.

These yield three pixel zones. The must-change zone is the union of regions of $A^{-}$. The may-change zone is the regions of $A^{+} \setminus A^{-}$. Everything else is the must-preserve zone. The rule language is monotone, so adding facts can only add conclusions. This guarantees $A^{-} \subseteq A^{+}$, and the bracket is well defined by construction.

Relation to Prior Work

The editing side of the literature motivates the gap. One group of methods enforces strict zero drift:

  • NEP regenerates only the tokens inside the user's mask, so background change is impossible by construction.
  • Background Consistency Guidance with Layer-wise Memory restores the original latents outside the mask at every step of multi-turn editing.

Another group documents the cost of that strictness:

- Image Editing As Programs reports ghost shadows and lighting mismatches, and adds an annular blending mask to hide seams.

  • CoCoEdit reports that rigid preservation produces unnatural compositions. It responds with RL post-training that rewards pixel similarity outside the mask, and it notes that multimodal LLM judges miss subtle background drift.

All four methods take the mask as given. None asks whether the mask is the right region. Our proposal leaves their generation machinery unchanged and replaces only the region they are told to edit.

Two papers already sit at the boundary between symbolic structure and editing, and they are the closest prior work:

- Image Editing As Programs uses GPT-4o to decompose an instruction into atomic region operations. It has no check that one operation does not invalidate another region, and nothing in the program represents shadows or support.

  • FoR-SALE rewrites spatial phrases from an object's own viewpoint into camera coordinates. It uses 32 hand-written template rules and acknowledges failures under multiple instances and occlusion.

We differ from both in what is rewritten. They rewrite the instruction. We rewrite the instruction together with the scene's dependency structure into an affected-region set, and that set is closed under rule application.

The reasoning literature supplies the method and also limits it:

  • SPARQL-OWL on HermiT decomposes complex query patterns into canonical axiom templates for cache lookup and hierarchy-based pruning. We reuse that move: compound instructions such as "move the lamp and remove the vase" become canonical templates like Remove(x), Move(x, r), and Relight(x), which are evaluated against the scene's fact base.
  • TrOWL distinguishes root unsatisfiability from errors derived from it. This maps directly onto our distinction between the edited object (the root modification) and its shadows and reflections (derived modifications).
  • The bracketing idea comes from sound relaxations (Haga et al.) and complete upper-bound compilations (Zhou et al.).

Bötcher et al. act as a constraint. They show that approximating logics with inverse or functional roles can require infinite or non-elementarily large rule sets. We therefore never approximate an expressive spatial logic. We design directly in a Horn fragment with no functional roles, evaluated over finite scene graphs of tens of objects. Rule evaluation is then polynomial and, in practice, takes milliseconds.

Research Program

Perception layer. This layer turns an image into a scene graph: object instances with masks, plus typed relation facts with confidences. The relations are shadowOf, reflectionOf, supportedBy, occludes, containedIn, and illuminates. We assemble this from off-the-shelf segmentation, shadow and reflection detection, and depth estimation. The confidence threshold that separates certain from possible facts is a single tuned parameter.

Rule layer. This is a Horn rule set of about 20 to 40 rules, written once and not per image. Representative rules:

  • Any entity that is a shadow or reflection of an affected entity is itself affected.
  • Objects supported by a moved or removed object are affected under Move and Remove.
  • Regions occluded by a removed object must be in-filled, so they are affected.
  • Under Relight, regions illuminated by a light source are affected.

Instructions are parsed into canonical edit templates. A language model may do this parsing, but only into a closed template vocabulary, so the step can be checked. After each edit turn, the fact base is updated. Later edits then inherit dependencies created by earlier ones. For example, if a lamp added in turn one is moved in turn three, its light pool moves with it. This is the dependency analogue of the Layer-wise Memory latent log.

Execution layer. Zone-aware execution is applied to a fixed mask-conditioned editor:

  • The must-preserve zone is hard-locked.
  • The must-change zone is edited freely.
  • The may-change zone is generated under a background-consistency penalty of reduced strength, following the BCG mechanism.

We will run this on two backbones: a NEP-style region-restricted autoregressive editor and a FLUX inpainting editor. Running both shows that the gain comes from the mask, not from the generator.

Audit output. Every changed pixel can be traced to a derivation, for example "changed because shadowOf(s, cup) and Remove(cup)." This explanation is useful beyond the metrics.

Baselines

The naive reference point is the editor run with the user's or benchmark's object mask alone, hard-locked outside. This is the strict zero-drift regime of NEP and BCG. It should score very well on background preservation and poorly on dependent-region correctness, and it fixes one end of the trade-off curve.

The strongest existing method on preservation is CoCoEdit applied to Qwen-Image-Edit, which reports +2.8 dB PSNR on unedited regions of GEdit-Bench over the untuned model. On MagicBrush single-turn editing, the strongest number in our file is NEP's L1 of 0.0547 and DINO similarity of 0.9044. Both are evaluated against the standard object masks. The comparison therefore has to be extended to our two-sided metrics, not just quoted. We will run CoCoEdit-tuned Qwen-Image-Edit on the same images and score it with the same zone-aware metrics. To beat it, we must match its PSNR in the must-preserve zone within 0.5 dB while significantly lowering the residual-artifact rate in dependent regions.

The program-based baseline is Image Editing As Programs with its annular compositing mask. Its reported GPT-4o faithfulness is 4.41/5. We run it with the same FLUX backbone.

The dilation baseline expands the object mask by a swept radius. We report the full curve of drift against residual artifacts, so that our method is compared with the best dilation, not an arbitrary one.

The key ablation is a VLM-predicted dependency mask. GPT-4o, or an open VLM, receives the same perceived scene graph and is asked to list affected entities directly, with no rules. Perception, executor, and zone handling are identical. This isolates the contribution of rule-based closure. A second ablation collapses the bracket by treating all facts as certain. This tests whether the may-change zone matters.

All methods share the same images, the same executor checkpoint where applicable, and the same perception outputs. Each receives an equal tuning budget of 50 configurations on a held-out validation split.

Why not just…?

Why not just dilate the mask, as the annular mask in Image Editing As Programs effectively does? Dilation frees a fixed band around the object. Cast shadows under low sun, reflections in mirrors or water, and objects resting on a moved table can lie far outside any small band. A large band, meanwhile, frees unrelated nearby content, which is exactly the leakage we want to prevent. This is a real threat on images where all dependents are adjacent to the object. The dilation sweep will show directly whether any radius matches our zones. We expect dilation to be competitive on a "near-dependent" subset and to fail on a "far-dependent" subset, and we report the two subsets separately.

Why not just ask a VLM which regions the edit affects? This is the most serious alternative. The ablation above tests it with everything else held fixed. CoCoEdit's finding that multimodal LLM judges miss fine background inconsistencies suggests VLMs will be weak at exhaustive enumeration, and the ghost shadows reported for Image Editing As Programs suggest the same. The rule layer's advantage should appear as transitive dependencies (a cup on a tray on a moved table, plus the cup's shadow) and as consistency across multi-turn edits. If the VLM matches the rules on both, we will say so. The remaining contribution would then be determinism and auditable derivations, not accuracy.

Why not just train a model to predict the full change region from paired data, or rely on CoCoEdit-style RL? CoCoEdit's rewards are defined relative to a given mask. A wrong mask is therefore rewarded for being preserved, and RL cannot correct the mask itself. A learned change-region predictor is plausible, but existing training sets such as UltraEdit's 131K masked pairs annotate the edited object, not its dependents. Supervision for dependent regions would have to be built first, and the benchmark subset we annotate could seed it. We include a learned predictor trained on our annotations as an exploratory comparison and report its data efficiency against the rules.

Won't logic make editing slow? No, provided the rules stay in a Horn fragment. The worst-case blowups of Bötcher et al. concern approximating expressive ontologies, which we never do. Over finite scene graphs, forward chaining is cheap. Perception dominates the added cost, and we report it against NEP's 2.88 s per edit.

Proposed Experiments

Data. We take 600 single-turn edits from GEdit-Bench, ImgEdit-Bench, and MagicBrush that involve object removal, addition, movement, or relighting. For each, annotators mark dependent regions (shadows, reflections, supported objects, revealed occlusions). The result is a three-way ground truth of object, dependents, and independent background. We stratify the edits into near-dependent and far-dependent subsets. For multi-turn evaluation, we use Multi-Edit Bench and MagicBrush sessions, adding dependency annotations at each turn. We prioritize sessions where a later turn touches an object introduced or affected earlier.

Metrics. Each metric targets one of the two errors, plus edit quality:

  • Leakage: PSNR and SSIM in the ground-truth independent background.
  • Ghost artifacts: a blind human judgment of whether residual shadows, reflections, or floating supported objects remain, measured as residual-artifact rate in ground-truth dependent regions.
  • Edit quality: instruction faithfulness by human rating and by the benchmarks' own editing scores, so that gains in preservation are not bought with under-editing.

Decisive checks. There are three:

  1. At matched independent-background PSNR, dependency masks give a lower residual-artifact rate than the best dilation radius and than CoCoEdit on the far-dependent subset. If not, the core claim fails.
  2. Rule-derived masks beat the VLM-predicted dependency mask on transitive and multi-turn cases. If not, the logic contributes only auditability.
  3. Collapsing the bracket raises either leakage or artifacts. If not, the soundness/completeness framing is decoration and should be dropped.

We will also measure how perception errors propagate. We inject controlled false and missing relation facts and test whether errors stay confined to the may-change zone, as the bracket predicts.