Abstract
Cook's (Cook, 1977) distance is one of the most important diagnostic tools for detecting influential individual or subsets of observations in linear regression for cross-sectional data. However, for many complex data structures (e.g., longitudinal data), no rigorous approach has been developed to address a fundamental issue: deleting subsets with different numbers of observations introduces different degrees of perturbation to the current model fitted to the data and the magnitude of Cook's distance is associated with the degree of the perturbation. The aim of this paper is to address this issue in general parametric models with complex data structures. We propose a new quantity for measuring the degree of the perturbation introduced by deleting a subset. We use stochastic ordering to quantify the stochastic relationship between the degree of the perturbation and the magnitude of Cook's distance. We develop several scaled Cook's distances to resolve the comparison of Cook's distance for different subset deletions. Theoretical and numerical examples are examined to highlight the broad spectrum of applications of these scaled Cook's distances in a formal influence analysis.
Introduction
In statistical modeling, influence analysis is critical for identifying data points or clusters that disproportionately affect model parameters. The standard metric for this is Cook’s distance (CD), which quantifies the change in parameter estimates when specific observations are removed. While highly effective for independent, cross-sectional data, CD encounters significant challenges when applied to complex data structures, such as longitudinal studies.
In longitudinal data, researchers often delete entire clusters (e.g., all observations for a single subject). A fundamental problem arises: deleting a cluster with 20 observations naturally causes a larger "perturbation" to the model than deleting a cluster with only two. Because standard Cook’s distance does not account for this size differential, it inherently flags larger clusters as influential, regardless of whether they are truly outliers. This paper introduces a "scaled" Cook’s distance to provide a fair comparison across varying subset sizes.
Methodology
Measuring Perturbation
The authors propose a rigorous framework to normalize influence diagnostics. The first step involves measuring the degree of perturbation, denoted as $P(I|\mathcal{M})$, introduced by deleting a subset $I$. This is calculated using the Kullback–Leibler (KL) divergence, which measures the information loss between the original model and the model post-deletion. This provides a objective metric for how much "pressure" a deletion puts on the model structure.
Stochastic Ordering and Scaling
The research mathematically demonstrates that as the degree of perturbation increases, the Cook’s distance also tends to increase stochastically. This confirms that the "size issue" is not a random occurrence but a structural property of the CD metric itself.
To rectify this bias, the authors define Scaled Cook’s Distances ($SCD_1$ and $SCD_2$). These metrics normalize the raw Cook’s distance by its expected mean and standard deviation (or median) under the assumption that the model is correctly specified. This allows researchers to determine if a subject is more influential than expected for a cluster of its specific size.
Computational Efficiency
Calculating these metrics for complex models, such as Generalized Linear Mixed Models (GLMMs), can be computationally prohibitive if it requires re-fitting the model hundreds of times. To address this, the authors provide first-order Taylor approximations. These approximations allow for rapid estimation of scaled distances, making the method practical for large-scale datasets where raw parametric bootstrapping would be too slow.
Experimental Evaluation
Simulated Longitudinal Data
The method was tested on 100 simulated datasets generated from a linear mixed model. The simulation intentionally varied cluster sizes and introduced specific influential clusters. The results confirmed the problem: standard Cook’s distance showed a 0.83 correlation with cluster size, meaning it acted primarily as a proxy for the amount of data deleted rather than the presence of anomalies.
Yale Infant Growth Data
The authors applied their method to the Yale Infant Growth Data, which tracks the weight of 298 children (3,176 total observations) to study the effects of prenatal cocaine exposure. This dataset is a prime candidate for SCD because the number of visits per child varies significantly, ranging from 2 to 30.
Under standard CD analysis, subjects with the most visits dominated the list of influential points. However, after applying the Scaled Cook’s Distance, the authors identified Subject 246 as highly influential. Despite having a moderate raw CD, Subject 246 was a significant outlier relative to their cluster size—a discovery that the unscaled metric failed to highlight.
Discussion and Results
The findings demonstrate that SCD successfully identifies influential clusters that standard CD misses, particularly small clusters with high leverage. Furthermore, the distribution of scaled distances can serve as a diagnostic tool for the model itself. If a model shows an excessive number of influential points after scaling, it may indicate that the model is misspecified or has failed to capture the underlying correlation structure of the data.
Conclusion
This paper provides a necessary correction to a long-standing issue in influence diagnostics. By accounting for the degree of perturbation through scaling, researchers can perform fairer and more accurate data audits in complex longitudinal and hierarchical models. Future work may extend these frequentist diagnostics into Bayesian frameworks and apply them to survival models or datasets with missing covariate information.
Summary updated