Preventing Color Bleeding in Generative Image Synthesis

37 concepts37 papers2 proposals

When a neural network colorizes a black-and-white photo or generates an image from a sketch, it faces a fundamental spatial challenge: it must map low-frequency chromatic information onto high-frequency structural boundaries. In standard convolutional architectures, downsampling layers aggregate global context but discard precise spatial coordinates. When the network upsamples these features to predict pixel-level colors, the lack of sharp spatial constraints causes hues to spill across edges—a phenomenon known as color bleeding or chromatic leakage.

Understanding why color bleeding happens, and how modern architectures prevent it, requires looking at how networks balance semantic understanding with local geometric precision.

Why Networks Bleed Color

In automated colorization, a model typically predicts chrominance channels (such as $a^*b^*$ in $L^*a^*b^*$ space) given a luminance channel ($L$). As noted by Larsson et al. in their work on Learning Representations for Automatic Colorization, dense pixel-level predictions in cluttered scenes frequently fail to respect the boundaries of distinct objects.

Without an explicit mechanism to align color transitions with intensity gradients, the network treats color as a continuous field rather than a bounded property of an object. This issue is compounded in user-guided systems. For instance, Sangkloy et al. observed in Scribbler: Controlling Deep Image Synthesis with Sketch and Color that sparse user color strokes tend to leak into adjacent semantic regions because the network fails to strictly isolate color propagation within complex scene boundaries. Similarly, Zhao et al. encountered boundary-crossing color bleeding in Legacy Photo Editing with Learned Noise Prior when models were presented with ambiguous or conflicting user hints.

Semantic and Object-Level Parsing

The most direct way to keep color inside its lines is to teach the network what it is colorizing. If a model can explicitly segment an object, it can treat the object's boundary as a hard stop for color propagation.

In Pixelated Semantic Colorization, Zhao et al. addressed this by jointly training a model to perform pixel-level semantic segmentation alongside colorization. By forcing the network to recognize object edges explicitly, the model gains the spatial awareness needed to prevent chromatic values from spilling into the background.

Taking this further, Jin et al. focused on fine-grained human parsing in Focusing on Persons: Colorizing Old Images Learning from Modern Historical Movies. By explicitly segmenting clothing items from skin tones, their framework prevents garment dyes from bleeding into human skin. To bridge the gap between global texture and entity boundaries, Pucci et al. introduced a collaborative framework in Collaborative Image and Object Level Features for Image Colourisation that pairs standard CNNs with capsule networks. Because capsule networks excel at modeling object hierarchies and entity-level semantics, they provide the structural scaffolding necessary to anchor colors to specific objects.

Decoupling Color Selection via Palettes and Attention

Instead of forcing a single network to simultaneously decide which colors to use and where to put them, a more robust strategy separates color selection from spatial distribution.

In PalGAN: Image Colorization with Palette Generative Adversarial Networks, Xia et al. implemented a two-stage process. The model first generates a global color palette representing the scene's overall color scheme. Then, a specialized "chromatic attention" module maps these palette colors to individual pixels. Because the attention mechanism explicitly correlates color transitions with the sharp edges and intensity gradients of the original grayscale image, it drastically reduces bleeding artifacts.

This palette-guided approach has also been successfully adapted to generative diffusion models. Mao et al. demonstrated in Exploring Palette based Color Guidance in Diffusion Models that integrating palette representations directly into the diffusion denoising process enforces a tighter coupling between spatial segmentation and color guidance, preventing hues from drifting across structural boundaries.

Structural Regularization and Interactive Constraints

When automated semantic understanding is insufficient—such as in highly detailed line art or scenes with adjacent, texturally identical objects—researchers use structural regularization or interactive user constraints to enforce boundaries.

For line art, where pigments must be strictly confined within thin geometric lines, Mingcheng Yuan introduced a Concatenation and Spatial Attention (CSAt) module in Line Art Colorization With Concatenated Spatial Attention. This module dynamically weighs spatial features to ensure that user-provided hints do not spill over high-frequency line boundaries in complex areas like hair or eyes.

When multiple distinct objects share similar textures, models often suffer from "color collapse," where a single color bleeds across all instances. To solve this, Lee et al. developed a transformer-based framework in Enabling Region-Specific Control via Lassos in Point-Based Colorization. By allowing users to draw a rough "lasso" around an object, the model treats the lasso as a soft spatial constraint, preventing the color from intermingling with adjacent, semantically identical entities.

Finally, boundary alignment can be enforced mathematically through structural regularization layers. Xiong et al. demonstrated in Boosting the Performance of Generic Deep Neural Network Frameworks with Log-supermodular CRFs that appending a log-supermodular Conditional Random Field (CRF) as a trainable layer forces dense predictions to maintain local consistency. This explicit structural regularizer ensures that output transitions are mathematically aligned with the underlying structural edges of the input image, providing a generalizable defense against attribute bleeding.

Key Foundational Work

Colorization Using Quaternion Algebra with Automatic Scribble Generation (2012) — contributed 2 concept elements to this direction.
Text Extraction from Web Images Based on Human Perception and Fuzzy Inference (2001) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_color_bleeding_c4b8c8f9,
  title = {color bleeding},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/color-bleeding/rdr_c4b8c8f98d264459b960cfb239862a77},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_color_bleeding_c4b8c8f9_v1,
  title = {color bleeding},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/color-bleeding/rdr_c4b8c8f98d264459b960cfb239862a77/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake