Flexible Heteroscedastic Count Regression with Deep Double Poisson Networks

Spencer Young, Porter Jenkins, Longchao Da, Jeffrey Dotson, Hua Wei

Abstract

Predicting discrete counts with neural networks often involves a trade-off between using continuous distributions that predict impossible fractional values or discrete distributions that restrict uncertainty estimates. We introduce the Deep Double Poisson Network (DDPN), a model that predicts discrete counts while allowing for fully heteroscedastic uncertainty. By combining discrete inductive biases with unrestricted variance, DDPN achieves state-of-the-art accuracy, calibration, and out-of-distribution detection across multiple data modalities including tabular, image, text, and point cloud data.

Introduction

In real-world machine learning applications, it is insufficient for a model to provide only a point prediction; it must also quantify its uncertainty. For continuous regression tasks, neural networks typically output the mean and variance of a Gaussian distribution, enabling heteroscedasticity—the capacity to predict varying levels of uncertainty based on the input. However, many practical tasks involve count regression, where the target variable is a non-negative integer (e.g., crowd sizes, hospital stay durations, or star ratings).

Existing approaches for count data often fall short. Gaussian models assign probability mass to impossible negative or fractional values, which degrades calibration. Conversely, standard discrete distributions like Poisson or Negative Binomial (NB) impose rigid mathematical constraints on the relationship between mean and variance. A Poisson distribution forces the variance to equal the mean, while a Negative Binomial distribution requires the variance to exceed the mean (over-dispersion). These constraints prevent models from expressing arbitrary levels of uncertainty, particularly under-dispersion, which is critical for robust predictive modeling.

The Deep Double Poisson Network (DDPN)

The proposed Deep Double Poisson Network (DDPN) addresses these limitations by utilizing the Double Poisson distribution. The neural network outputs two parameters: $\mu$ (mean) and $\gamma$ (inverse dispersion). This formulation effectively decouples the mean from the variance, allowing the model to learn unrestricted uncertainty for discrete counts.

A significant methodological contribution is the concept of learnable loss attenuation. The authors demonstrate that DDPN naturally downweights the loss of extreme outliers; if a data point is difficult to fit, the model can predict high uncertainty ($\gamma$), preventing noisy samples from biasing the overall model. To mitigate potential "lazy" convergence where the model over-relies on uncertainty, the authors introduce $\beta$-DDPN. This variant uses a tunable hyperparameter $\beta$ to prioritize mean accuracy during the early stages of training. Additionally, Deep Ensembles of DDPNs are employed to capture both aleatoric uncertainty (inherent data noise) and epistemic uncertainty (model ignorance).

Experimental Evaluation

DDPN was evaluated across four diverse datasets representing different modalities:

  • Length of Stay (Tabular): Predicting hospital stay duration based on patient health metrics.
  • COCO-People (Image): Counting individuals in images from the MS-COCO dataset.
  • Inventory (Point Cloud): Counting retail products from 3D point cloud data.
  • Amazon Reviews (Text): Predicting discrete 1-5 star ratings from text reviews.
  • Out-of-Distribution (OOD) Detection: Testing the Amazon Review model's ability to identify Bible verses as OOD data.

Key Insights and Results

DDPN serves as a "Goldilocks" solution for count regression, offering the flexibility of Gaussian models without violating the discrete, non-negative nature of the data. A key insight is that constrained variance in standard discrete models (Poisson/NB) corrupts ensemble uncertainty estimates. Because these base models cannot accurately capture aleatoric noise, the resulting ensemble's measure of epistemic uncertainty—often calculated as the disagreement between models—becomes unreliable.

Comparison of Poisson, Negative Binomial (NB), and DDPN ensembles on synthetic count data.

DDPN and its $\beta$-variant outperformed all baselines, including Poisson, NB, and various Gaussian configurations, in both Mean Absolute Error (MAE) and Continuous Ranked Probability Score (CRPS) across all datasets. DDPN ensembles also demonstrated superior performance in OOD detection. On the text dataset, DDPN achieved an AUROC of 0.887 for flagging Bible verses, significantly outperforming Gaussian (0.840) and standard Poisson (0.330) models.

Conclusion

DDPN provides a robust and flexible framework for heteroscedastic count regression. While its benefits are most pronounced for low-to-medium frequency counts where discreteness is critical, it offers a principled approach to uncertainty quantification that surpasses traditional discrete and continuous baselines. Future research will focus on extending DDPN's OOD detection capabilities to other modalities and refining optimization strategies for the Double Poisson's negative log-likelihood.

Summary updated

BibTeX
@inproceedings{young2025flexible,
  author = {Spencer Young and Porter Jenkins and Longchao Da and Jeffrey Dotson and Hua Wei},
  title = {{Flexible Heteroscedastic Count Regression with Deep Double Poisson Networks}},
  booktitle = {International Conference on Machine Learning},
  year = {2025},
  url = {https://openreview.net/forum?id=1WWqIuph7j},
}
Figure