Exhaustive Goodness of Fit Via Smoothed Inference and Graphics

Sara Algeri, Xiangyu Zhang

Abstract

Classical tests of goodness of fit aim to validate the conformity of a postulated model to the data under study. Given their inferential nature, they can be considered a crucial step in confirmatory data analysis. In their standard formulation, however, they do not allow exploring how the hypothesized model deviates from the truth nor do they provide any insight into how the rejected model could be improved to better fit the data. The main goal of this work is to establish a comprehensive framework for goodness of fit which naturally integrates modeling, estimation, inference and graphics. Modeling and estimation focus on a novel formulation of smooth tests that easily extends to arbitrary distributions, either continuous or discrete. Inference and adequate post-selection adjustments are performed via a specially designed smoothed bootstrap and the results are summarized via an exhaustive graphical tool called CD-plot. Technical proofs, codes and data are provided in the supplementary material.

Introduction

In classical statistics, goodness-of-fit (GoF) tests like Pearson’s $\chi^2$ or Kolmogorov-Smirnov act as "black boxes." They provide a p-value to reject or fail to reject a null hypothesis but offer little insight into the nature of the model's failure. This strictly confirmatory approach lacks exploratory power, failing to identify whether discrepancies occur in the tails, the mean, or due to outliers. Furthermore, many traditional "smooth tests" are distribution-specific and struggle with discrete data or small sample sizes where asymptotic assumptions are unreliable. This paper introduces a unified framework for "exhaustive" GoF testing that integrates modeling, estimation, inference, and graphics to not only detect model failure but also diagnose and correct it.

Methodology: LP Score Functions and Valid Estimators

The proposed framework, implemented in the R package LPsmooth, relies on three technical pillars. First, it utilizes LP score functions as a universal system of orthonormal bases. This allows for a consistent treatment of both continuous and discrete data by working in the quantile domain via the Probability Integral Transform ($U = G(X)$).

Second, the framework addresses the issue of "non-bona fide" densities. When expanding a model to improve fit, simple polynomial expansions (like Barton's) can result in negative values or densities that do not integrate to one. The authors incorporate the Gajek estimator, which ensures the resulting model is a mathematically valid probability distribution.

Comparison of different comparison density estimators, including the Gajek estimator used in this framework to ensure valid probability distributions.

Third, for robust inference, the authors employ an LP-smoothed bootstrap and a Bidirectional Acceptance Sampling algorithm. This approach avoids reliance on shaky large-sample asymptotics and significantly accelerates simulations by sampling from the hypothesized and estimated models simultaneously.

Graphical Diagnostics: The CD-plot

A central innovation of this work is the Comparison Density (CD) plot. The CD-plot visualizes the ratio $d(G(x), G, F)$ between the true data distribution and the hypothesized model. A perfect fit results in a flat line at 1. Deviations—peaks or troughs—indicate exactly where the model over- or under-estimates the data.

Top: Probability and density estimates for discrete and continuous data. Bottom: Corresponding CD-plots showing how the Gajek estimator provides a valid, smoothed fit compared to empirical or Barton estimates.

As shown in Figure 2, the CD-plot (bottom row) provides a clear visual diagnostic. For discrete data (left), the step-function nature of the empirical CD-plot is smoothed by the LP-based estimators. For continuous data (right), the CD-plot highlights how the initial parametric model deviates from the true distribution, allowing for automatic model correction.

Empirical Evaluation and Results

The framework was validated using both simulated and real-world data:

  • Simulated Data: Tests on Binomial and truncated Normal distributions demonstrated the tool's ability to pinpoint specific deviations in both discrete and continuous settings.
  • COVID-19 Hospitalization Data: Analyzing the time from symptom onset to hospitalization for 572 patients, the authors initially tested a Beta-Negative Binomial (BNB) model. The deviance test rejected the BNB ($p=0.0018$). The CD-plot revealed the cause: an "excess of zeros" (immediate hospitalizations). By identifying this specific failure, the authors were able to recover a better-fitting Zero-Inflated Negative Binomial (ZINB) model, which the framework subsequently failed to reject ($p=1$).

Conclusion

The LPsmooth framework transforms goodness-of-fit from a binary decision into an informative diagnostic process. By providing automatic model correction and intuitive graphical tools like the CD-plot, it bridges the gap between confirmatory and exploratory data analysis. While currently focused on univariate data, it offers a powerful toolkit for modern statistical modeling, with future work aiming to extend these tools to multivariate and regression contexts.

Summary updated

Abstract

Classical tests of goodness of fit aim to validate the conformity of a postulated model to the data under study. Given their inferential nature, they can be considered a crucial step in confirmatory data analysis. In their standard formulation, however, they do not allow exploring how the hypothesized model deviates from the truth nor do they provide any insight into how the rejected model could be improved to better fit the data. The main goal of this work is to establish a comprehensive framework for goodness of fit which naturally integrates modeling, estimation, inference and graphics. Modeling and estimation focus on a novel formulation of smooth tests that easily extends to arbitrary distributions, either continuous or discrete. Inference and adequate post-selection adjustments are performed via a specially designed smoothed bootstrap and the results are summarized via an exhaustive graphical tool called CD-plot. Technical proofs, codes and data are provided in the supplementary material.
BibTeX
@article{algeri2021exhaustive,
  author = {Sara Algeri and Xiangyu Zhang},
  title = {{Exhaustive Goodness of Fit Via Smoothed Inference and Graphics}},
  journal = {Journal of Computational and Graphical Statistics},
  year = {2021},
  doi = {10.1080/10618600.2021.1999826},
}
Figure