Abstract
The frailty model extends Cox’s proportional hazards model by incorporating a random effect to account for unobserved heterogeneity. While estimation algorithms like the EM algorithm were previously proposed, their theoretical properties remained unverified. This paper provides the first rigorous proof of the existence and consistency of the Nonparametric Maximum Likelihood Estimators (NPMLE) for the cumulative baseline hazard and the variance of the random effect.
Introduction
In survival analysis, the standard assumption is that individuals with the same observed characteristics share the same risk profile. However, unmeasured factors—such as genetic predisposition or shared environmental exposures—often create "frailty" within certain groups. The frailty model addresses this by adding a random effect, $Z$, to the hazard function. Susan A. Murphy’s 1994 paper, published in The Annals of Statistics, provides the necessary mathematical foundation to ensure that estimates derived from these models are reliable as sample sizes increase.
The Mathematical Framework
The paper focuses on the Gamma frailty model, where the random effect $Z$ follows a Gamma distribution with mean 1 and variance $\theta$. The hazard for an individual is modeled as $\lambda(t) = Z \lambda_0(t)$, where $\lambda_0(t)$ is the baseline hazard function.
Murphy treats the estimation as a Nonparametric Maximum Likelihood (NPMLE) problem. Unlike parametric models that assume a specific shape for the baseline hazard (e.g., Weibull), this approach allows the cumulative hazard function, $A(t)$, to be estimated as a step function. This flexibility makes the model more robust but significantly complicates the proof of consistency, as the parameter space for $A(t)$ is infinite-dimensional.
Proving Consistency
The core of the paper is the proof that as the number of groups $n \to \infty$, the estimators $(\hat{\theta}, \hat{A})$ converge to the true population parameters $(\theta_0, A_0)$. Murphy employs the counting process framework and the Innovation Theorem to handle the stochastic nature of survival data.
Key technical challenges addressed include:
- Identifiability: Confirming that the variance $\theta$ and the function $A$ can be uniquely determined from observed event times.
- Bounding the Likelihood: Using a comparison estimator to prove that the estimated cumulative hazard does not diverge to infinity as the sample size increases.
- Uniform Convergence: Proving consistency in the supremum norm, ensuring the estimated hazard function is accurate across the entire time range.
Practical Considerations
The paper includes a theoretical example involving left-truncated and right-censored data, demonstrating the model's applicability to complex clinical trial designs. A vital insight from the work is the requirement for "repeat" events; to estimate the variance of the frailty effect, there must be groups in the dataset that experience more than one event. Without this clustering information, the model cannot distinguish between group-level frailty and the general baseline risk.
Scope and Limitations
While this paper provides a foundational proof, it focuses on the "one-sample" case—groups of individuals without additional covariates. Murphy suggests the logic extends to regression settings, but that specific extension was not explicitly detailed here. Additionally, the proof relies heavily on the mathematical properties of the Gamma distribution; consistency for other distributions (like Inverse Gaussian or Log-Normal) requires separate investigation. Finally, the proof assumes a non-increasing "risk set," which may be violated in study designs where individuals enter and leave the study multiple times.
Conclusion
By providing a rigorous proof of consistency, Murphy’s work validated the use of the EM algorithm for frailty models. It ensured that as researchers collect more data, their insights into hidden heterogeneity and baseline risks converge toward the truth, solidifying the frailty model as a cornerstone of modern survival analysis.
Summary updated