CONGA: Compiling Gaussian Fiber Geometry for Structured Bayesian Inference

Víctor GallegoarXiv 2026 · arXiv

TL;DR

CONGA (CONditionally GAussian) is a compiler that automatically identifies and "whitens" large Gaussian latent blocks within structured Bayesian models. By analytically handling these Gaussian components, it reduces the inference task to a much smaller non-Gaussian "core," leading to speedups of over 12x and enabling convergence in high-dimensional problems where standard methods fail.

What Problem Does This Paper Address?

Many common Bayesian models—such as state-space models, Gaussian processes, and hierarchical models—possess a specific structure: a large block of latent variables $x$ is Gaussian when conditioned on a much smaller set of parameters $\eta$. The authors call this a Gaussian fiber.

Standard inference methods like Hamiltonian Monte Carlo (HMC) or Normalizing Flows are "geometry-blind." They spend a significant portion of their "warmup" phase trying to numerically re-learn this Gaussian geometry from scratch. In high-dimensional spaces, this is not just inefficient; it often leads to a total failure to converge because the sampler gets lost in the complex geometry of the posterior.

Methods

The authors introduce a workflow that treats inference as a compilation problem. The process, illustrated in Figure 1, follows these steps:

  1. Discovery & Validation: CONGA uses "numerical probes" (secants of the score function) to identify which coordinates are jointly quadratic (Gaussian). It then validates these candidates against the model's algebra to ensure the Gaussian identity holds.
  2. Exact Transports: Once a Gaussian fiber is confirmed, CONGA applies an exact affine transformation (such as a symmetric Brenier map or a triangular Knothe–Rosenblatt map). This mathematically "whitens" the fiber, turning it into a standard normal distribution that is independent of the non-Gaussian core.
  3. Graph-Based Compilation: The "compiler" aspect involves choosing the most efficient way to compute this transform based on the model's precision graph:
  4. Temporal: Uses sequential Kalman filters or balanced associative scans for time-series data.
  5. Spectral: Uses fast transforms (like the Discrete Cosine Transform) for regular grids/lattices.
  6. Sparse: Uses Cholesky-based methods for general sparse graphs.
  7. Inference on the Core: The sampler (e.g., NUTS) now only needs to explore the small, non-Gaussian core $\eta$. The large Gaussian block $x$ is handled analytically, significantly reducing the dimensionality of the learning problem.
Figure 1: The CONGA workflow. (a) A Gaussian fiber determines the exact family in Eq. (2); its Brenier and KR members share factorization (Proposition 1) and core-only learning (Corollary 2). (b) For an exact contract, CONGA chooses an implementation from the graph and the workload. Numerical candidates retain the target through Eq. (10); invalid or expensive candidates leave it unchanged.

Datasets

The authors evaluated CONGA across several challenging benchmarks:

  • Stock–Watson Inflation Model: A classic econometrics model for quarterly inflation with persistent and transient volatility.
  • Electricity-Load DLM: A massive regional electricity load model with 6,760 dimensions, reduced to a 34-dimensional core.
  • Graph-Indexed Fields: Synthetic benchmarks on various topologies, including trees, 2D lattices, planar graphs, expanders, and "star" graphs.
  • PosteriorDB: A standardized library of 46 different Bayesian models used to test CONGA's automatic discovery procedure.

Key Insights and Ideas

  • Separation of Concerns: Standardizing the Gaussian fiber doesn't just make the math cleaner; it fundamentally simplifies the optimization landscape. It reduces the Kullback–Leibler (KL) and Fisher divergence objectives, making it much easier for normalizing flows to learn the remaining distribution.
  • Implementation is Everything: The paper highlights that while different maps (Brenier vs. KR) might be mathematically equivalent, their computational costs vary wildly depending on the graph structure. For example, on a 2D lattice, a spectral map is significantly faster than a sparse Cholesky implementation.
  • Fail-Safe Mechanism: If CONGA cannot prove a Gaussian identity algebraically, it defaults to a "target-preserving change of variables." This ensures that even if the discovery is slightly off, the resulting inference remains valid and calibrated.

Key Results

  • Massive Efficiency Gains: On the Stock–Watson model, CONGA improved the minimum Effective Sample Size (ESS) per second by 12.5× for NUTS.
  • Restoring Convergence: In the 6,760-dimensional electricity model, raw NUTS failed to converge entirely (rank-$\hat{R}$ > 2.1). CONGA enabled perfect convergence ($\hat{R} \approx 1.006$) in comparable time.
  • Automatic Discovery: In a census of 46 models from PosteriorDB, CONGA successfully identified Gaussian fibers in 38 of them without any model-specific configuration.
  • Graph Performance: As shown in Figure 2, for most graph types (trees, lattices, expanders), the sparse Rao-Blackwellized NUTS enabled by CONGA significantly outperformed raw NUTS and even learned flows in terms of ESS per second.
Figure 2: Matched inference on graph fields at n = 256 (three-seed medians). Minimum core ESS/s and target-equivalent rows per ESS are shown only when every seed passes rank- bR ≤1.05; time includes discovery and reconstruction. Complete values and protocols are in Appendix B.1.

Limitations and Future Work

  • Symbolic Requirements: While discovery is numerical, full marginalization still requires an "algebraic contract" (symbolic proof) to be truly robust.
  • Template Scope: The current implementation focuses on temporal and graph-based templates. Extending this to arbitrary kernels or non-Gaussian fibers (e.g., discrete variables) remains a challenge.
  • Manual Intervention: Some complex templates, like balanced Gaussian message passing, are currently supplied by hand. The authors aim to automate the translation of recovered paths into these templates in future work.

Summary updated

Abstract

In structured Bayesian models, a large latent block is often Gaussian conditional on a smaller set of parameters, yet generic inference methods relearn this geometry during warmup. We study how this structure can be used to construct transports for posterior inference. The conditional mean and precision determine exact affine maps, including symmetric Brenier and triangular Knothe-Rosenblatt maps, that separate a standard normal block from the remaining non-Gaussian core. KL and relative Fisher objectives then reduce to objectives on this core. We introduce CONGA, a compiler that identifies candidate Gaussian blocks and selects a temporal, spectral, or sparse implementation according to their structure and computational cost. Marginalization requires an identity established from the model; numerical evidence alone permits only a target-preserving change of variables, and a failed check leaves the model unchanged. On the Stock-Watson inflation model, CONGA improves minimum ESS per second by 12.5x for NUTS. It also restores convergence where the raw target fails: for WALNUTS on the same model, and for NUTS on a 6760-dimensional electricity load model reduced to 34 core coordinates. Experiments on graph-indexed fields show how coloring and fill affect the choice of map, including cases where retaining the original target is preferable. Code at https://github.com/vicgalle/conga
BibTeX
@misc{gallego2026conga,
  author = {Víctor Gallego},
  title = {{CONGA: Compiling Gaussian Fiber Geometry for Structured Bayesian Inference}},
  year = {2026},
  eprint = {2611.20995},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.20995},
}
Figure