When merging two images—such as pasting a person into a new background or applying a painterly style to a photo—deep learning models often struggle with "feature bleeding." This occurs when the textures of the background leak into the foreground object, or when the identity of a reference object distorts the surrounding environment.
To solve this, researchers have moved away from global processing, instead using masked attention mechanisms to spatially decouple features during the generation process. By restricting where the model "looks" at each step, these methods ensure that foregrounds and backgrounds harmonize without losing their individual integrity.
The Problem of Feature Bleeding
In standard attention mechanisms, every pixel (or latent token) can potentially attend to every other pixel. While this global context is powerful, it is problematic for image composition. If you are trying to colorize a sketch using a reference image, a global attention mechanism might mistakenly "bleed" the color of a reference character's hair into the background of your target sketch. Dingkun Yan and Zhuoru Li refer to this as spatial entanglement, where misaligned reference features cause ghosting or structural artifacts in the final output.
Spatial-Separated Attention
The foundational intuition for solving this was introduced by Xiaodong Cun and Chi‐Man Pun through the Spatial-Separated Attention Module ($S^2AM$). Instead of treating the feature map as a monolithic block, they used a binary mask to partition it into distinct regions.
By applying specialized attention gates to the foreground and background independently, the model can selectively transform the style of a pasted object while preserving the integrity of the background. This ensures that the "harmony" of the composite image comes from matching the lighting and tone, rather than physically blurring the boundaries between the two sources.
Decoupling in Diffusion Models
As the field shifted toward diffusion models, the need for spatial control became even more acute. In tasks like virtual try-on, S.J. Wan and Siqi Wan developed GarDiff, which uses decoupled cross-attention. By applying a spatial mask to attention weights, the model restricts the influence of a reference garment to the specific target region on the person's body, preventing the texture of the clothes from bleeding into the person's skin or the background.
A similar approach is used in painterly harmonization, where a real-world object is inserted into a painting. Bo-Kai Ruan and Hong-Han Shuai introduced the Similarity Disentangle Mask in their TF-GPH framework. This mask ensures that:
- The foreground region attends to the foreground reference for identity and shape.
- The composite region attends to the background reference for stylistic elements (like brushstrokes).
This spatial redirection prevents "content disruption"—such as a face becoming distorted because it tried to attend to the abstract textures of a Van Gogh background.
Multi-Concept Composition and Optimal Transport
When an image contains multiple distinct objects (e.g., a specific dog and a specific toy), the risk of "concept bleeding" increases. Hao Fu and Tianyu Su address this in OTComp by using spatial masks to gate the flow of information from multiple reference sources.
By combining these masks with Dual Optimal Transport, the model first establishes a coarse structural alignment and then injects high-frequency textures only where the mask allows. This hierarchy ensures that the dog doesn't end up with the texture of the toy, even if they are physically overlapping in the composition.
Why It Works: Selective Influence
The success of these methods lies in the realization that attention is a routing mechanism. In a standard transformer, information flows freely based on feature similarity. By introducing a spatial mask into the attention calculation—essentially multiplying the attention matrix by a mask before the softmax or during the weighting phase—researchers force the model to ignore irrelevant "keys" and "values."
Whether it is the split cross-attention used by Dingkun Yan for anime colorization or the forgery-detection-based masks used by Qi Zhang, the principle remains the same: spatial disentanglement is achieved by making the model's receptive field semantically aware. This allows for high-fidelity image composition where different regions can follow different rules, styles, and reference sources simultaneously.