Fully Heteroscedastic Count Regression with Deep Double Poisson Networks

Spencer Young, Porter Jenkins, Longchao Da, Jeff Dotson, Hua Wei

TL;DR

The authors introduce the Deep Double Poisson Network (DDPN), the first neural count regression model to achieve "full heteroscedasticity"—the ability to model input-dependent variance without being restricted by the mean. By leveraging the Double Poisson distribution, DDPN provides superior accuracy, better-calibrated uncertainty, and state-of-the-art out-of-distribution detection across tabular, image, text, and point cloud data.

What Problem Does This Paper Address?

In deep learning, we often care as much about how sure a model is as we do about the prediction itself. For continuous data (like predicting house prices), we use "heteroscedastic Gaussian" models that predict both a mean and a variance. This allows the model to account for aleatoric uncertainty (noise inherent in the data).

However, for count regression (predicting integers like the number of people in a crowd or days in a hospital), we’ve been stuck. Standard tools like Poisson models force the variance to equal the mean, while Negative Binomial models force the variance to be greater than or equal to the mean. Using Gaussian models for counts is a "misspecification" nightmare: they predict impossible negative values and non-integers, leading to poor calibration. The lack of a flexible, discrete model has made it difficult to get reliable uncertainty estimates for count-based AI systems.

Methods

The core innovation is the Deep Double Poisson Network (DDPN). Instead of predicting a single parameter, the network outputs two: $\mu$ (the mean) and $\gamma$ (the inverse dispersion).

DDPN Architecture

Key technical components include:

  • Double Poisson Distribution: Unlike Poisson or Negative Binomial, this distribution can represent under-dispersion (variance < mean), equi-dispersion (variance = mean), and over-dispersion (variance > mean).
  • Learnable Loss Attenuation: The authors prove that DDPN's loss function naturally allows the model to "ignore" outliers by increasing predicted uncertainty—a property previously thought to be unique to Gaussian models.
  • $\beta$-DDPN: To prevent the model from getting "stuck" by explaining away all hard-to-fit data as high noise, they introduce a $\beta$ hyperparameter. This allows practitioners to tune the "attenuation strength," balancing the focus between fitting the mean and estimating the variance.
  • Deep Ensembles: By ensembling multiple DDPNs, the authors capture epistemic uncertainty (uncertainty due to lack of model knowledge), which is significantly improved when the underlying members are fully heteroscedastic.

Datasets

The authors evaluated DDPN across four distinct modalities to prove its versatility:

  1. Length of Stay (Tabular): Predicting hospital stay duration (Microsoft dataset).
  2. COCO-People (Image): Counting people in images from the MS-COCO dataset using a ViT backbone.
  3. Inventory (Point Cloud): Counting retail products on shelves from 3D point clouds.
  4. Reviews (Text): Predicting 1-5 star ratings from Amazon reviews using a DistilBERT backbone.
  5. OOD Detection: Using Amazon Reviews as "in-distribution" and verses from the King James Bible as "out-of-distribution" data.

Key Insights and Ideas

  • Full Heteroscedasticity is the "Secret Sauce": The paper demonstrates that if individual ensemble members are constrained in their variance (like Poisson/NB), the entire ensemble’s uncertainty becomes miscalibrated. DDPN’s flexibility fixes this "leakage" between aleatoric and epistemic uncertainty.
Uncertainty Separation in DDPN vs Baselines
  • Discrete Inductive Bias Matters: Gaussian models fail on counts because they waste probability mass on "impossible" values (like 2.5 people). DDPN concentrates its probability mass on integers, leading to much sharper and more interpretable predictive intervals.
  • Robustness to Misspecification: Even when the data is actually generated by a Poisson or Negative Binomial process, DDPN recovers the true distribution as well as (or better than) models specifically designed for those distributions.

Key Results

  • State-of-the-Art Performance: DDPN (specifically the $\beta$-DDPN variant) outperformed all baselines in both MAE (Mean Absolute Error) and CRPS (Continuous Ranked Probability Score) across all real-world datasets.
  • Superior Calibration: In the "COCO-People" task, DDPN achieved an MAE of ~1.00 compared to ~1.10 for Poisson and ~1.22 for Gaussian models.
  • OOD Detection: DDPN ensembles significantly outperformed others in identifying out-of-distribution text. It achieved an AUROC of 0.887, compared to 0.330 for Poisson and 0.840 for Gaussian baselines. DDPN models "revert to the mean" (predicting high uncertainty) when they see data they don't recognize, whereas Poisson/NB models often remain overconfident.

Limitations and Future Work

  • High-Frequency Counts: The authors note that for very high counts (e.g., millions), the discrete advantage of DDPN might diminish, and a Gaussian approximation might be computationally cheaper and "good enough."
  • Moment Approximations: The model relies on Efron’s approximate moments for the Double Poisson. While the paper shows these are highly accurate for most ranges, they can deteriorate at extremely low means with very high variance.
  • OOD Scope: The OOD experiments were primarily focused on text; further research is needed to see if these robust OOD properties hold as strongly in high-dimensional image or point cloud spaces.

Summary updated

Abstract

Neural networks capable of accurate, input-conditional uncertainty representation are essential for real-world AI systems. Deep ensembles of Gaussian networks have proven highly effective for continuous regression due to their ability to flexibly represent aleatoric uncertainty via unrestricted heteroscedastic variance, which in turn enables accurate epistemic uncertainty estimation. However, no analogous approach exists for $\textit{count}$ regression, despite many important applications. To address this gap, we propose the Deep Double Poisson Network (DDPN), a novel neural discrete count regression model that outputs the parameters of the Double Poisson distribution, enabling arbitrarily high or low predictive aleatoric uncertainty for count data and improving epistemic uncertainty estimation when ensembled. We formalize and prove that DDPN exhibits robust regression properties similar to heteroscedastic Gaussian models via learnable loss attenuation, and introduce a simple loss modification to control this behavior. Experiments on diverse datasets demonstrate that DDPN outperforms current baselines in accuracy, calibration, and out-of-distribution detection, establishing a new state-of-the-art in deep count regression.
BibTeX
@inproceedings{young2025fully,
  author = {Spencer Young and Porter Jenkins and Longchao Da and Jeff Dotson and Hua Wei},
  title = {{Fully Heteroscedastic Count Regression with Deep Double Poisson Networks}},
  booktitle = {International Conference on Machine Learning},
  year = {2025},
  url = {https://openreview.net/forum?id=1WWqIuph7j},
}
Figure