Recursive Bayesian Inference via Functional Conjugacy

This direction is merged into another direction: conjugate prior optimization. Its last version stays readable and citable here.

Version 1 of 115 papers19 conceptsfrozen 2026-10-04

Bayesian inference is often described as a process of "updating" beliefs: you start with a prior, observe data, and arrive at a posterior. In practice, however, this update is rarely a clean hand-off. For most complex models, the posterior distribution becomes mathematically more complicated than the prior, requiring expensive numerical methods like MCMC to solve.

A growing body of research focuses on functional conjugacy—the property where the posterior distribution belongs to the same mathematical family as the prior. When this property holds, the Bayesian update simplifies from a complex integration problem into a simple algebraic update of parameters. This allows for recursive filtering, where the output of one update becomes the input for the next, enabling real-time inference in dynamic environments.

The Algebra of Belief Updating

The most intuitive form of this idea appears in discrete settings. As Gwendolyn M. Eadie, Daniela Huppenkothen, and Aaron Springford demonstrate in their educational work, using a conjugate prior (like the Beta distribution for Bernoulli trials) ensures the posterior remains within that same family.

This isn't just a pedagogical convenience; it is a computational engine. Chen Ma explains in The Hitchhiker's Guide to LDA that the Dirichlet-Multinomial conjugacy is what makes Latent Dirichlet Allocation (LDA) scalable. By keeping the prior and posterior in the same family, researchers can derive analytical solutions that bypass the need for heavy simulations. Anirban Bhattacharya and colleagues further prove that this Dirichlet-Multinomial relationship allows Bayesian models to achieve optimal convergence rates without the technical hurdles typically found in non-parametric statistics.

Maintaining Consistency in Sequential Streams

The true power of conjugacy emerges in sequential tasks, such as radar tracking or change-point detection. If the posterior distribution changes its functional form at every step, the model quickly becomes unmanageable.

To solve this, Cong Peng and Wei Yi derive principled conjugate priors for radar systems. By ensuring the posterior remains in the same family, they create a "recursive loop" where the system can track multiple targets indefinitely without ad-hoc approximations. Similarly, Lorena Romero-Medrano and Antonio Artés-Rodríguez use Dirichlet-Multinomial conjugacy to perform online change-point detection. Because the updates are closed-form, the algorithm can process high-dimensional data streams in real-time.

Expanding the Family: Mixtures and Sparsity

Standard conjugate families are often too simple to capture real-world complexity, such as multi-modal data or sparsity. Recent work has expanded the definition of these "families" to include mixtures.

Jeffrey P. Spence introduced a technique using mixtures of non-overlapping exponential families. He proved that if the individual components of a mixture (like a point mass at zero and a Gaussian) are conjugate to a likelihood, the entire mixture remains conjugate. This allows variational inference to capture complex structures—like whether a variable is exactly zero—while maintaining the speed of analytic updates.

Amortized and Approximate Conjugacy

The most recent frontier involves using neural networks to "learn" these conjugate mappings. Juliusz Krzysztof Ziomek and Michael A. Osborne introduced Distribution Transformers (DTs) to bridge the gap between deep learning and Bayesian rigor.

In the DT framework, distributions are represented as Gaussian Mixture Models (GMMs). A Transformer architecture is trained to map an input prior (represented as GMM parameters) to an output posterior. Crucially, Ziomek et al. and George Whittle enforce a state of approximate conjugacy: the output posterior is forced into the same GMM format as the input prior.

This architectural constraint allows the Transformer to function as a recursive filter. Because the output "looks like" the input, the model's own predictions can be fed back into itself for sequential data updates. This achieves the millisecond speeds of amortized deep learning while preserving the principled, recursive nature of classical Bayesian filters. As Michele Caprio and Eyke Hüllermeier note in their work on upper probabilities, preserving these structural properties (like concavity or functional form) is essential for the stability and consistency of uncertainty modeling over time.

Papers in this version

History & citationVersion 1 · 2026-10-04

Merged into another direction · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_recursive_bayesian_inference_via_functio_b7fb0201,
  title = {Recursive Bayesian Inference via Functional Conjugacy},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/recursive-bayesian-inference-via-functional-conjugacy/rdr_b7fb020107964b5abac19293725a38b4},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_recursive_bayesian_inference_via_functio_b7fb0201_v1,
  title = {Recursive Bayesian Inference via Functional Conjugacy},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/recursive-bayesian-inference-via-functional-conjugacy/rdr_b7fb020107964b5abac19293725a38b4/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}