Unbiased Gradient Estimation for Discrete Latent Variables

21 concepts12 papers1 proposal

Standard deep learning relies on the smooth flow of gradients through continuous functions. However, many real-world problems require discrete decisions: selecting a word from a vocabulary, turning a neuron on or off, or assigning a data point to a specific cluster. These "staircase" functions have zero gradients almost everywhere and infinite gradients at the jumps, making them incompatible with standard backpropagation.

The Limits of Continuous Relaxation

The traditional way to handle discrete variables is to "relax" them into continuous approximations. For example, a binary choice might be replaced by a value between 0 and 1 using a softmax or a Concrete distribution. While these relaxations allow for gradient descent, they introduce a fundamental discrepancy between training and inference.

Li et al. argue that these biased relaxations often lead to "mathematical drift," where the model performs well during training but fails when the discrete gates are actually applied. This is particularly evident in complex tasks like neural network pruning, where biased estimators struggle to maintain performance at high sparsification rates. Similarly, Dong et al. note that unbiased estimators eliminate this training-inference gap by ensuring the model remains consistent with its discrete operational state throughout the optimization process.

Unbiased Stochastic Estimators

To avoid the pitfalls of relaxation, researchers have developed estimators that provide unbiased gradient signals directly for discrete variables. A prominent example is the Augment-Reinforce-Merge (ARM) estimator.

The intuition behind ARM is to use antithetic sampling—evaluating the model at two related points—to cancel out the high variance typically associated with the REINFORCE algorithm. Boluki et al. demonstrate that ARM provides a low-variance, unbiased signal for Bernoulli variables, allowing for the stable training of learnable dropout rates. By treating binary masks as stochastic variables rather than fixed hyperparameters, the authors achieve superior predictive performance without the instabilities of continuous approximations.

This approach extends to structured tasks as well. Min et al. utilize unbiased estimators to model human gaze as a discrete latent variable, finding that this principled approach leads to more stable attention maps in activity recognition compared to biased heuristics.

Deterministic and Functional Alternatives

While stochastic estimators focus on sampling, another branch of research seeks to bypass sampling altogether through deterministic proxies or functional optimization.

Wiseman proposes replacing the standard Evidence Lower Bound (ELBO) with an objective based on the Bethe free energy. This provides a deterministic approximation of the partition function, allowing for exact gradient calculations in structured models like Hidden Markov Models (HMMs) without the noise of sampling. Wiseman et al. further show that reframing the learning of Markov Random Fields as a saddle-point problem enables stable end-to-end training by backpropagating through predicted marginals.

Other methods move away from gradients of the likelihood entirely. Crucinio introduces a mirror descent approach that decouples the parameter update from the latent variable representation. This allows for the optimization of models with non-differentiable latent spaces, such as graphs or mixture models, where traditional gradient-based methods fail.

Principled Discrete Representations

The shift toward unbiased and principled estimation is also transforming how we learn discrete representations like codebooks and hashes.

In the context of Vector Quantized VAEs (VQ-VAEs), Takida et al. introduce SQ-VAE, which replaces heuristic "straight-through" estimators with a formal Variational Bayes framework. By using stochastic quantization, the model naturally transitions from exploration to exploitation, effectively eliminating the common problem of "codebook collapse."

For retrieval tasks, Wang introduces BerVAE, which treats video hash codes as probabilistic Bernoulli variables. By using unbiased gradient estimators instead of deterministic "hacks," the model learns more distinct data clusters and provides a reliable measure of uncertainty for each generated bit.

Ultimately, these diverse methods share a common goal: moving beyond the "smoothness" requirement of deep learning to treat discrete choices as first-class citizens in the optimization process. By prioritizing unbiased signals and principled objectives, these approaches enable more robust, interpretable, and efficient models across a wide range of applications.

Key Foundational Work

Amortized Bethe Free Energy Minimization for Learning MRFs (2019) — contributed 2 concept elements to this direction.
Direct Optimization through arg max for Discrete Variational Auto-Encoder (2019) — contributed 2 concept elements to this direction.
Learning Deep Latent-variable MRFs with Amortized Bethe Free Energy Minimization (2019) — contributed 2 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_unbiased_discrete_gradient_estimation_64c97617,
  title = {unbiased discrete gradient estimation},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/unbiased-discrete-gradient-estimation/rdr_64c97617fc554cdfbc9b2f9e4195b3ed},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_unbiased_discrete_gradient_estimation_64c97617_v1,
  title = {unbiased discrete gradient estimation},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/unbiased-discrete-gradient-estimation/rdr_64c97617fc554cdfbc9b2f9e4195b3ed/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake