Preventing Color Bleeding in Generative Colorization

This direction is split into other directions: color bleeding. Its last version stays readable and citable here.

Version 1 of 193 papers99 conceptsfrozen 2026-10-04

The fundamental challenge in image colorization is not just choosing the right hue, but keeping it where it belongs. Whether a model is colorizing a historical photograph, a hand-drawn sketch, or a video frame, it must reconcile two often-conflicting signals: high-level semantic intent (e.g., "this is a red car") and low-level structural boundaries (the sharp edge where the car meets the road).

When these signals are poorly integrated, the result is color bleeding—a phenomenon where pigments "leak" across object boundaries, creating unnatural halos, ghosting, or chromatic contamination. Recent research has shifted from simple color "hallucination" toward sophisticated spatial grounding techniques that treat colorization as a boundary-constrained optimization problem.

The Mechanism of Leakage

Color bleeding typically occurs when a model fails to distinguish between pixels that are spatially adjacent but semantically distinct. In early optimization-based methods, this was often a result of using standard Euclidean distances for color propagation, which struggle to respect complex textures. Xujie Li and Hanli Zhao addressed this in Local linear model optimization based grayscale image colorization by replacing Euclidean metrics with "diffusion distance" and the Matting Laplacian, which better aligns color flow with the underlying image structure.

In modern deep learning, the problem is often architectural. Generative models, particularly Latent Diffusion Models (LDMs), operate in a compressed mathematical space. As noted by Yanru An in the development of MT-Color, this compression can blur structural edges, causing colors to spill into the background. Similarly, Xiangcheng Du and colleagues argue in SeAda that a "semantic gap" exists where insufficient input information prevents the model from accurately mapping grayscale structures to the expected color domain.

Structural Guardrails and Edge Awareness

To prevent leakage, researchers have introduced "guardrails" that force models to respect intensity gradients. One classic approach is Side Window Filtering, which shifts the processing window so that the pixel sits on the edge or corner rather than the center. This allows the filter to "stay on one side" of a boundary, preventing information from bleeding across semantic regions.

More recent work leverages hierarchical relationships. Jin-Hwi Park and Jaesung Choe introduced the Hyperbolic Affinity learning Module (HAM), which uses hyperbolic geometry to model pixel relationships. By treating image features as a hierarchy, HAM can distinguish between pixels that are neighbors in a 2D grid but belong to different objects, significantly reducing bleeding errors at object boundaries.

Semantic Anchoring and Instance Control

A major breakthrough in preventing bleeding is the use of semantic guidance to "anchor" colors to specific regions. Instead of a global colorization pass, models like DDColor, developed by Xiaoyang Kang and Tao Yang, use a dual-decoder architecture. A pixel decoder handles spatial resolution while a query-based transformer decoder ensures semantic consistency. This separation allows the model to reconcile high-level "what" information with low-level "where" details.

For user-driven tasks, such as colorizing sketches with scribbles, the challenge is even greater. Patsorn Sangkloy and James Hays demonstrated in Scribbler that GANs can learn to "hallucinate" details while respecting sparse color strokes, though they still occasionally struggle with boundary precision in complex scenes.

To achieve finer control, researchers are now moving toward instance-aware strategies. Shuchen Weng and Boxin Shi proposed L-CAD, which uses instance-aware sampling to distinguish between multiple individual objects of the same type. By treating each object as a separate entity, these models prevent "color binding errors" where the color of one object (like a red shirt) accidentally bleeds into another (like a red car) simply because they share a semantic label.

From GANs to Diffusion

The field is currently transitioning from GAN-based architectures to Conditional Diffusion Probabilistic Models (DPMs). While GANs are fast, they can be unstable and prone to artifacts. Hernan Carrillo and Edgar Simo-Serra showed in Diffusart that diffusion models can produce higher-fidelity illustrations by using dual-conditioning strategies to keep colors strictly within the physical boundaries of line art.

However, even diffusion models require explicit spatial constraints. Frameworks like VanGogh by Zixun Fang integrate depth guidance and "luma replacement" to eliminate temporal flickering and bleeding in video. By incorporating geometric information (depth) alongside color, these models ensure that the structural integrity of the scene is maintained, even as the colors are synthesized from scratch.

Papers in this version

Showing 60 of 93 papers.

History & citationVersion 1 · 2026-10-04

Split into other directions · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_preventing_color_bleeding_in_generative__8176ed1f,
  title = {Preventing Color Bleeding in Generative Colorization},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/preventing-color-bleeding-in-generative-colorization/rdr_8176ed1ff0644674bd66a2a335f1a0dc},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_preventing_color_bleeding_in_generative__8176ed1f_v1,
  title = {Preventing Color Bleeding in Generative Colorization},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/preventing-color-bleeding-in-generative-colorization/rdr_8176ed1ff0644674bd66a2a335f1a0dc/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}