When we model multivariate data using Gaussian distributions, capturing dependencies is straightforward. Whether two variables move together (positive correlation) or in opposite directions (negative correlation), the underlying mathematics remains stable. This stability exists because the Gaussian distribution's density is governed by a quadratic form that naturally suppresses extreme values, provided the precision matrix is positive definite.
However, real-world data is rarely just continuous and symmetric. We frequently encounter count data (such as word frequencies or genomic sequencing reads) or durations (such as time-to-event data). To model these, we turn to univariate exponential families like the Poisson or Exponential distributions.
The challenge arises when we try to link these univariate distributions into an undirected graphical model—a Markov Random Field (MRF). As researchers discovered, naively combining non-Gaussian exponential families introduces a severe mathematical crisis: the joint distribution frequently becomes non-normalizable, meaning its total probability mass diverges to infinity.
The Normalizability Trap
To understand why these models blow up, consider a standard pairwise MRF. The joint probability distribution of a vector of variables $X = (X_1, X_2, \dots, X_p)$ is defined by exponentiating a sum of node-wise potentials and interaction terms:
$$P(X) \propto \exp \left( \sum_{i} \theta_i X_i + \sum_{i < j} \theta_{ij} X_i X_j \right)$$
To make this a valid probability distribution, the sum (or integral) over all possible states must equal a finite number. This normalizing constant is known as the partition function, $Z$. If $Z$ is infinite, the distribution is mathematically invalid.
Let’s look at what happens when $X_i$ and $X_j$ are Poisson-distributed counts, which have an infinite domain ($\mathbb{N} = \{0, 1, 2, \dots\}$). The inherent log-probability of a univariate Poisson distribution drops off because of a factorial term, which behaves roughly like $-X_i \log X_i$ in the exponent.
If we introduce a positive interaction coefficient ($\theta_{ij} > 0$) to represent that $X_i$ and $X_j$ tend to increase together, the interaction term contains the product $\theta_{ij} X_i X_j$. If we move along the diagonal where $X_i = X_j = x$, the positive interaction term grows quadratically as $\theta_{ij} x^2$.
Because a quadratic function ($x^2$) grows vastly faster than the suppressing base term ($x \log x$), the positive interaction completely overwhelms the distribution's tail decay. As $x$ goes to infinity, the terms inside the partition function sum grow larger and larger instead of shrinking to zero. The partition function explodes to infinity, rendering the joint distribution non-normalizable.
The Negative Dependency Straitjacket
Because of this explosion, early frameworks for generalized linear graphical models were forced to accept a highly restrictive compromise. As shown by Yang et al. 2012 and Yang et al. 2015, to guarantee a finite partition function over an infinite domain, the interaction parameters $\theta_{ij}$ must be strictly non-positive ($\theta_{ij} \le 0$).
This mathematical constraint forces a severe practical limitation: standard Poisson or Exponential MRFs can only model negative conditional dependencies. They can capture competitive or inhibitory relationships—where the presence of one variable suppresses another—but they are fundamentally incapable of modeling positive correlations, such as words that frequently co-occur in a document or genes that are co-expressed.
This constraint is not unique to graphical models; theoretical frameworks analyzing dependent variables across statistics have frequently found themselves restricted to negative dependence structures to maintain mathematical tractability, as noted by Wang et al. 2011. Conversely, frameworks designed purely for positive dependence, such as Multivariate Total Positivity, suffer from the opposite problem—they cannot accommodate competitive or inhibitory relationships (Wang et al. 2020, Yan et al. 2025).
To build truly expressive models for non-Gaussian data, researchers had to find a way to break out of this negative dependency straitjacket.
Solution 1: Domain Truncation
The most direct way to prevent a distribution from exploding at infinity is to prevent it from reaching infinity in the first place.
In Yang et al. 2013 ("On Poisson Graphical Models"), the authors introduced Truncated Poisson Graphical Models. By capping the domain of each count variable at a fixed upper bound $R$ (so that $X_i \in \{0, 1, \dots, R\}$), the state space becomes finite. Because the sum of any finite number of terms is always finite, the partition function can never diverge. This simple truncation completely frees the interaction parameters, allowing $\theta_{ij}$ to be positive or negative.
Domain truncation becomes even more critical when building Mixed Graphical Models, where different types of distributions (like Gaussian, Poisson, and Bernoulli) are combined into a single network. As Yang et al. 2014 demonstrated, naively mixing continuous infinite-domain variables with count variables causes the joint density integral to diverge. Truncating the domain of the count variables acts as a vital stabilizing mechanism, ensuring that interactions between disparate data types remain mathematically sound.
Solution 2: Sublinear Interaction Terms
While truncation works well in practice, it alters the underlying univariate distribution and requires choosing an arbitrary clipping threshold $R$. What if we want to preserve the true, infinite domain of the exponential family while still allowing positive dependencies?
The root of the problem is that the standard interaction term $X_i X_j$ grows too quickly. To fix this, Inouye et al. 2016 proposed Square Root Graphical Models (SQR). Instead of multiplying the raw variables, the SQR framework applies a square root to the sufficient statistics in the interaction term:
$$\text{Interaction} = \theta_{ij} \sqrt{X_i} \sqrt{X_j}$$
By slowing down the growth of the interaction term, the mathematics changes completely. Along the diagonal $X_i = X_j = x$, the interaction term now grows linearly ($\theta_{ij} x$) rather than quadratically. Because the negative base log-probability of the Poisson distribution grows as $-x \log x$, it easily dominates the linear interaction term as $x \to \infty$.
The tail of the distribution decays safely to zero, ensuring the partition function remains finite even when $\theta_{ij} > 0$. This allows the model to capture positive dependencies naturally without bounding the data. A similar philosophy was explored by Yang et al. 2013 ("On Poisson Graphical Models") through the introduction of Sublinear and Quadratic PGMs, which similarly modify the energy function to guarantee normalizability over infinite domains.
Alternative Architectures for Count Data
Altering the energy function of an MRF is not the only way to bypass normalizability constraints. Other lines of research have abandoned the traditional undirected graphical model structure entirely to achieve the same goal.
One approach is to change how the network is trained and specified. Molina et al. 2015 introduced Poisson Dependency Networks, which avoid specifying a joint MRF altogether. Instead, they learn a set of local conditional distributions using gradient-boosted trees. Because dependency networks do not rely on a global partition function, they can seamlessly capture both positive and negative correlations across count data.
Another approach leverages deep tractability. Molina et al. 2017 developed Poisson Sum-Product Networks (PSPNs). PSPNs are deep hierarchical architectures that represent joint distributions through network layers of sums (mixtures) and products (independent factorizations) of univariate Poisson distributions. Because the network structure guarantees that the joint distribution is always a valid, multi-modal mixture of independent components, it bypasses the normalizability issues of MRFs entirely, allowing for exact inference of complex positive and negative dependencies in linear time.
Key Foundational Work
Graphical Models via Generalized Linear Models (2012) — contributed 3 concept elements to this direction.
On the exponential inequality for acceptable random variables (2011) — contributed 1 concept elements to this direction.