Bernoulli Diffusion for Discrete Binary Data

This direction is split into other directions: Bernoulli noise modeling. Its last version stays readable and citable here.

Version 1 of 115 papers22 conceptsfrozen 2026-10-04

Standard diffusion models, like those powering Stable Diffusion, operate in a continuous world. They add small amounts of Gaussian noise to an image until it becomes a blur, then learn to reverse that blur. But much of the data we care about—segmentation masks, graph edges, or user-item interactions—is fundamentally discrete. When we force these binary "bits" into a continuous Gaussian framework, we introduce a mathematical mismatch: the model must learn to map continuous values back to discrete ones, often leading to instability and artifacts.

A growing body of research suggests a more natural alternative: Bernoulli Diffusion. Instead of blurring data with Gaussian noise, these models "corrupt" data by flipping bits.

The Forward Process: From Signal to Bit-Flips

In a standard diffusion model, the forward process $q(x_t | x_{t-1})$ adds Gaussian noise. In Bernoulli diffusion, the forward process is defined by a Bernoulli transition kernel. As Ze Wang and Ze Wang 0008 describe in Binary Latent Diffusion, this means that at each step, a bit has a certain probability of flipping its state (from 0 to 1 or vice versa).

As $t$ increases, the bits are flipped more frequently until the original structure is lost and the data reaches a stationary distribution—typically a uniform distribution where each bit has a 50% chance of being 0 or 1. This "bit-flip" noise is mathematically native to binary data. There is no need for thresholding or rounding because the data remains in its discrete state $\{0, 1\}$ throughout the entire process.

The Reverse Process: Denoising as Binary Classification

The magic of Bernoulli diffusion happens in the reverse process. Instead of predicting a "noise vector" to subtract, the model learns to predict the probability that a bit in its noisy state was originally a 0 or a 1.

Harshit Varma and colleagues demonstrate in the Glauber Generative Model that this simplifies the generative task into a series of binary classification problems: "Is this bit signal or noise?" By leveraging Glauber dynamics—a concept from statistical physics—they show that this approach can outperform complex variational methods.

Why It Works: Mathematical Alignment

The primary advantage of this approach is alignment. In medical imaging, for instance, Chenhui Wang and Hongming Shan argue in HiDiff that using a Bernoulli-based kernel prevents the degradation of binary structures. When refining a segmentation mask, a Gaussian model might produce "gray" intermediate values that don't correspond to any physical reality. A Bernoulli model, however, works directly with the discrete nature of the mask, leading to sharper boundaries and more stable training.

This alignment is equally critical in non-visual domains:

  • Combinatorial Optimization: In DIFUSCO, Zhiqing Sun and Yiming Yang use Bernoulli noise to solve problems like the Traveling Salesman Problem. Flipping an edge from "included" to "excluded" aligns perfectly with the discrete decision-making required for graph-based tasks.
  • Recommendation Systems: Gabriel Bénédict and the team behind RecFusion found that for sparse user-item interaction vectors, "directional" noise—flipping active bits to inactive—creates a smoother learning gradient for the model to reconstruct missing interactions.

Efficiency and Representation

Beyond accuracy, Bernoulli diffusion offers significant efficiency gains. Rong Xiao and Shaozhe Hao show in BiGR that by using compact binary latent codes instead of continuous ones, models can unify high-quality image generation with robust visual representation learning.

Because binary latent spaces are much smaller than continuous ones, these models can often sample much faster. For example, the Binary Latent Diffusion framework can generate high-resolution 1024x1024 images in as few as 16 steps, a massive leap over the hundreds of steps often required by traditional Gaussian diffusion models.

Selective Refinement

Finally, Bernoulli diffusion allows for "surgical" edits. In Uncertainty-Masked Bernoulli Diffusion, Yuqi Shen and Fengyang Xiao use the process to refine camouflaged object detection. By combining uncertainty quantification with Bernoulli noise, the model identifies only the "unsure" parts of a mask and applies bit-flips there, selectively repairing the segmentation without disturbing the parts the model is already confident about.

Papers in this version

History & citationVersion 1 · 2026-10-04

Split into other directions · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_bernoulli_diffusion_for_discrete_binary__1af3350f,
  title = {Bernoulli Diffusion for Discrete Binary Data},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/bernoulli-diffusion-for-discrete-binary-data/rdr_1af3350f418c4578b43a3cdeec1608ff},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_bernoulli_diffusion_for_discrete_binary__1af3350f_v1,
  title = {Bernoulli Diffusion for Discrete Binary Data},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/bernoulli-diffusion-for-discrete-binary-data/rdr_1af3350f418c4578b43a3cdeec1608ff/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}