Abstract
This paper extends the classical Owen's table of normal integrals to higher dimensions, providing elegant closed-form solutions for complex integrals involving the product of multiple Gaussian cumulative distribution functions (CDFs) and a Gaussian density. Leveraging these mathematical identities, the author derives a novel multivariate Bernoulli distribution that offers a clean, latent-variable approach to modeling correlated binary data.
Introduction
In fields such as Bayesian statistics and machine learning—particularly in Gaussian process classification—researchers frequently encounter integrals that combine Gaussian densities with Gaussian CDFs (the "probit" link). While Owen’s 1980 table of normal integrals is a foundational reference for solving these, it is often limited to simpler, lower-dimensional cases. When these integrals lack known analytical solutions, researchers typically resort to numerical quadrature methods, which become computationally expensive and less accurate as dimensionality increases. This work aims to bridge this gap by providing exact formulas that bypass the need for approximation in specific multivariate settings.
Methodology
The paper is primarily a theoretical derivation employing several key mathematical techniques:
- Algebraic Transformation: The product of univariate Gaussian CDFs is rewritten as an integral over a latent space.
- Matrix Identities: To solve resulting multivariate forms, the author utilizes the partitioned matrix inversion lemma and the matrix determinant lemma, converting complex precision matrices into intuitive covariance structures.
- Fubini-Tonelli Theorem: This theorem justifies interchanging the order of integration, which is the critical step in simplifying an $N$-dimensional integral into a single evaluation of a multivariate Gaussian CDF.
The Multivariate Bernoulli Distribution
The core contribution is a new multivariate Bernoulli model. It begins with a latent Gaussian vector $\mathbf{f}$. Given $\mathbf{f}$, binary outcomes are assumed to be independent Bernoulli trials with probabilities defined by the probit link $\Phi(f_r)$. By integrating out the latent vector $\mathbf{f}$, the author identifies the exact marginal distribution of the binary outcomes.
Key Results
Analytical Identities
The paper establishes two primary lemmas. Lemma 2 provides a general formula for the integral of a product of $N$ univariate Gaussian CDFs against a Gaussian density: $$\int_{\mathbb{R}^N} \prod_{r=1}^N \Phi\left(\frac{x_r - m_r}{v_r}\right) \mathcal{N}(\mathbf{x} | \boldsymbol{\mu}_N, \Sigma) d\mathbf{x} = F_N(\boldsymbol{\mu}_N | \mathbf{m}_N, \text{diag}(\mathbf{v}_N)^2 + \Sigma)$$ This result is a powerful tool for probit modeling, expressing the integral as a single $N$-dimensional Gaussian CDF ($F_N$).
Probability Mass Function
Theorem 1 defines the marginal distribution for a vector of correlated binary variables $\mathbf{Y}$: $$\pi_{\mathbf{Y}}(y_1, \dots, y_N) = F_N(\mathbf{0} | -I_y \boldsymbol{\mu}, I_y \Sigma I_y + I_N)$$ where $I_y$ is a diagonal matrix of the outcomes. By using $\{-1, 1\}$ instead of $\{0, 1\}$ for outcomes, the author produces a symmetric and mathematically elegant probability mass function (PMF).
Discussion and Limitations
Insights
- Latent Correlation: The model allows for a flexible correlation structure. By selecting the covariance matrix $\Sigma$ of the underlying latent Gaussian process, one can precisely control dependencies between binary variables.
- Symmetry: The use of $\{-1, 1\}$ encoding results in a PMF that is simply the multivariate Gaussian CDF evaluated at zero, with mean and covariance adjusted by observed values.
Limitations
- Computational Complexity: While the solution is "closed-form," it relies on the multivariate Gaussian CDF ($F_N$). Evaluating $F_N$ for high dimensions (e.g., $N > 20$) remains a non-trivial challenge requiring specialized algorithms like Genz's method.
- Practical Application: As a theoretical work, it does not yet demonstrate performance on real-world tasks (e.g., spatial modeling) compared to existing methods like the Multivariate Logit or Ising models. Future work should explore implementation in probabilistic programming frameworks to assess speed and stability in Bayesian inference.
Summary updated