Many scientific comparisons ask which interactions between variables change between two conditions. Examples are gene co-expression in stimulated versus resting cells, or co-occurring diagnoses in two patient cohorts. When the data are counts, two separate literatures meet without talking to each other. One literature, direct density ratio estimation, estimates the change itself. It fits the ratio $p(x)/q(x)$ of the two distributions and never fits either one alone, so interactions that stay the same cancel out and only the sparse change has to be learned. The other literature, on normalizability, shows that the natural count models, Poisson Markov random fields, stop being valid probability distributions as soon as any interaction is positive. A Markov random field here means a graphical model whose density is the exponential of a sum of pairwise interaction terms. We argue that the same problem quietly breaks density ratio estimation on counts. If the change between conditions includes a strengthened positive interaction, the ratio model with the usual product features has an infinite average under the reference condition. The estimator's sample-based normalizer then has unbounded variance.
This note proposes to make the ratio itself normalizable. The ratio is built on the square-root sufficient statistics of Poisson square root graphical models, and we study what this buys in theory and in practice. The two-sentence version is as follows. Density ratio change detection on counts needs the ratio's second moment under the reference data to be finite, and standard product-of-counts features violate this for any positive change. Square-root features satisfy it for every change, which should give stable and provably consistent detection of both strengthened and weakened count interactions without fitting either network.
Core Claim
The direct change learner of Support Consistency of Direct Sparse-Change Learning in Markov Networks uses the Kullback-Leibler Importance Estimation Procedure (KLIEP). KLIEP models the ratio as $r(x) \propto \exp(\Delta^\top \phi(x))$, where $\phi$ is a vector of sufficient statistics and $\Delta = \theta_P - \theta_Q$ is the parameter difference. It replaces the intractable normalizer with an average over $n_q$ reference samples:
$$\widehat{\Delta} = \arg\min_\Delta \; -\frac{1}{n_p}\sum_{i=1}^{n_p} \Delta^\top \phi(x_p^{(i)}) + \log \frac{1}{n_q}\sum_{k=1}^{n_q} \exp\big(\Delta^\top \phi(x_q^{(k)})\big) + \lambda \|\Delta\|_1 .$$
The guarantees in that paper rest on well-behaved sufficient statistics. With Poisson-like reference tails, which decay roughly like $\exp(-x\log x)$, and the standard pairwise feature $x_s x_t$, the quantity $\mathbb{E}_q[\exp(c\,\Delta_{st} x_s x_t)]$ is infinite for every $\Delta_{st} > 0$ and every $c > 0$. The reason is that the quadratic exponent beats the $x\log x$ decay along the diagonal, which is the same mechanism that forces $\theta_{st} \le 0$ in Graphical Models via Generalized Linear Models and On Poisson Graphical Models. The population objective is undefined, and the empirical normalizer is driven by a few extreme cells.
Replace $\phi$ with the features of Square Root Graphical Models: $\sqrt{x_s x_t}$ for pairs and $x_s$ for single nodes. The exponent now grows at most linearly in the counts, so $\mathbb{E}_q[r(x)^k]$ is finite for every $k$ and every finite $\Delta$. We call this property ratio normalizability: the ratio has finite moments under the reference distribution.
Our claim has three parts. First, ratio normalizability is the right condition for density-ratio change learning on unbounded domains. Second, it holds for square-root statistics with no sign constraint on $\Delta$. Third, under it the support consistency argument of Liu et al. goes through for count data. A further hypothesis, which we will test rather than assert, is that the light tails of $r$ under $q$ relax the quadratic reference-sample requirement $n_q = \Omega(n_p^2/d)$ reported in that paper, where $d$ is the number of changed edges.
Research Program
The theoretical part has three steps. First, we characterize which feature growth rates make $\mathbb{E}_q[\exp(\Delta^\top\phi)]$ and its square finite under a reference distribution with Poisson-type base measure. Square-root and logarithmic features both qualify; product features do not. Second, we re-derive the primal-dual witness proof of Support Consistency of Direct Sparse-Change Learning in Markov Networks using sub-exponential concentration of the normalizer in place of boundedness assumptions, and we track how the resulting $n_q$ requirement depends on $\|\Delta\|$. Third, we state when $\Delta$ is interpretable. If both conditions are Poisson square root models, $\Delta$ is exactly the difference of their interaction matrices. Each of those matrices may be dense and positive, and neither is ever estimated.
On the method side, the estimator is the objective above with square-root features, solved by proximal gradient. Because the ratio is normalizable in both directions, we also fit $q/p$ and combine the two supports. This gives a cheap, symmetric answer to the reference-choice problem raised in Learning sparse structural changes in high-dimensional Markov networks. The combination is a secondary contribution. Ground-truth simulation needs samples from Poisson square root models, which we draw by Gibbs sampling over the node-wise conditionals, as in the square-root paper.
Baselines
The naive reference point is to transform counts with $\log(1+x)$ and run DIFFEE from Fast and Scalable Learning of Sparse Changes in High-Dimensional Gaussian Graphical Model Structure. DIFFEE is a closed-form Gaussian differential estimator that soft-thresholds covariances and subtracts proxy inverses. It is what a practitioner would do today. It receives the same data and the same grid size for its two thresholds.
The strongest existing competitor for count data is two-step PLNet from Estimating graphical models for count data with applications to single-cell gene network, fitted per condition and subtracted. PLNet is a Poisson log-normal model with a D-trace loss. On its own single-network simulation (random graph, $n=2000$, $p=100$, high dropout), it reports an area under the precision-recall curve (AUPR) of 0.88, against 0.69 for a variational baseline and 0.18 for graphical lasso. That number is for single-network recovery. We found no reported result on Lacuna for differential count-network recovery by any method. Beating this baseline means higher AUPR on the changed edges, at matched total sample size and matched regularization grid, in settings where each individual network is dense.
We also run standard KLIEP with product features on raw counts, as in Learning sparse structural changes in high-dimensional Markov networks. On Gaussian Markov networks with $p=50$ that paper reports an area under the ROC curve (AUC) of about 0.88 for KLIEP, while covariance-precision matching falls from 0.81 to 0.56.
The key ablation holds the estimator, penalty and tuning fixed and changes only $\phi$. It compares product features, product features on counts clipped at $R$, $\log(1+x_s)\log(1+x_t)$, and $\sqrt{x_s x_t}$. This isolates whether the gain comes from ratio normalizability or from the density-ratio formulation itself.
Why not just…?
Why not just log-transform and use a Gaussian differential method? PLNet's simulations show graphical lasso on count data collapsing to an AUPR of 0.18, against 0.88 for a count-aware model. That evidence is for single networks. Our DIFFEE-on-$\log(1+x)$ baseline measures directly whether the same failure carries over to differences. If it does not, the practical case for the proposal narrows to settings with heavy zero-inflation.
Why not just fit two count models and subtract? The subtraction trap has been documented for Gaussian models. Errors compound, and sparsity penalties bias estimates of dense networks, as reported in DiNeR and in the KLIEP review's comparison against covariance-precision matching. Two-step PLNet has never been tested in the differential setting. Our dense-background simulations will show whether it degrades as background density rises while the change stays sparse.
Why not just use any bounded or sublinear feature, such as clipping or $\log(1+x)$ products? This is the real threat. Our own theory says log features are also ratio-normalizable, and clipping makes everything finite. The proposal does not claim that square roots are uniquely stable. It claims that ratio normalizability is the operative condition. The feature ablation settles which normalizable choice detects changes best. If log features match square roots, we will report that.
There is one difference beyond stability. Square-root features make $\Delta$ an exact parameter difference between two valid joint models that allow positive dependencies. The clipping threshold $R$, by contrast, is known to change recovered structure substantially in On Poisson Graphical Models.
Proposed Experiments
The first setting is synthetic pairs of Poisson square root models with $p$ from 50 to 200. The backgrounds are dense or hub-dominated, and the changes add or strengthen positive edges as well as negative ones. We report AUPR and AUC on the changed edges.
The second setting is a two-condition version of the PLNet simulator, in which data come from a Poisson log-normal model with dropout. This tests robustness when our model is misspecified.
The third setting is the CD14+ monocyte single-cell data used by PLNet: 2,147 interferon-beta-treated cells and 200 genes, with differential edges checked against the ChIP-seq-derived hTFtarget database. The file describes only the treated cells. Running this experiment therefore requires a matched unstimulated population. If none is available, we fall back to comparing the miRNA networks in TCGA breast cancer subgroups and assess the results qualitatively against the hub miRNAs reported by Yang et al.
The decisive check fixes $n_p$ and $d$ and sweeps $n_q$ downward. It records support recovery and the effective sample size of the normalizer weights, $(\sum_k w_k)^2/\sum_k w_k^2$ with $w_k = \exp(\Delta^\top\phi(x_q^{(k)}))$. Product features should show weight collapse and failure whenever positive changes are present. Square-root features should keep stable weights and recover the support at smaller $n_q/n_p^2$ ratios.
A second decisive check sweeps background density. It should show the direct estimator's accuracy staying flat while two-step PLNet degrades. If either prediction fails, the central claim fails with it.
References
- Support Consistency of Direct Sparse-Change Learning in Markov Networks
- Graphical Models via Generalized Linear Models
- On Poisson Graphical Models
- Square Root Graphical Models: Multivariate Generalizations of Univariate Exponential Families that Permit Positive Dependencies
- Learning Sparse Structural Changes in High-dimensional Markov Networks: A Review on Methodologies and Theories
- Fast and Scalable Learning of Sparse Changes in High-Dimensional Gaussian Graphical Model Structure
- Estimating graphical models for count data with applications to single-cell gene network
- DiNeR: a Differential graphical model for analysis of co-regulation Network Rewiring