Abstract
In modern statistical analysis, researchers frequently encounter growing-dimensional data where the true likelihood function is unavailable. This paper introduces a Penalized Exponentially Tilted Likelihood (PETL) framework designed for variable selection and parameter estimation in growing-dimensional unconditional moment models. The PETL approach addresses challenges such as variable correlation and model misspecification. Under specific regularity conditions, we establish the consistency and oracle properties of PETL estimators and demonstrate that the constrainedly PETL ratio statistic follows an asymptotic central chi-squared distribution. Our findings indicate that PETL is significantly more robust to model misspecification than traditional empirical likelihood methods. The methodology is validated through extensive simulation studies and an application to the Boston Housing Study.
Introduction
As data collection capabilities expand, statisticians often face "growing-dimensional" scenarios where the number of parameters ($p$) increases alongside the sample size ($n$). A significant hurdle in these contexts is the absence of a well-defined likelihood function. Instead, researchers often rely on unconditional moment models—equations that describe the expected behavior of data without assuming a specific probability distribution.
While Empirical Likelihood (EL) has traditionally been the tool of choice for such models, it possesses a critical vulnerability: extreme sensitivity to model misspecification. If the moment conditions are even slightly inaccurate, EL performance degrades. Furthermore, high correlation between variables and diverging parameter counts often lead to slow convergence or failure in identifying significant variables. This paper proposes the PETL framework to provide a robust, high-dimensional inference tool capable of handling complex dependencies and misspecified models.
The PETL Framework
The proposed Penalized Exponentially Tilted Likelihood (PETL) integrates several advanced statistical components to ensure robustness and accuracy:
- Exponential Tilting (ET): Unlike the multinomial approach used in Empirical Likelihood, ET minimizes the Kullback-Leibler divergence between empirical frequencies and weights. This fundamental shift makes the framework inherently more resilient when the underlying model is misspecified.
- SCAD Penalty: To facilitate automatic variable selection, the authors incorporate the Smoothly Clipped Absolute Deviation (SCAD) penalty. This allows the model to shrink the coefficients of non-influential variables to zero, effectively identifying the sparse structure of high-dimensional data.
- Sieve Method: To manage the "ill-posed" nature of diverging parameters, a sieve approach is employed to constrain the parameter space, ensuring the estimator remains consistent even as dimensions grow.
- Asymptotic Theory: The paper rigorously proves that the PETL estimator possesses "oracle properties," meaning it performs as efficiently as if the true set of relevant variables were known a priori. Additionally, the authors demonstrate that a version of Wilks’ Theorem holds, allowing for the use of standard chi-squared tests in hypothesis testing.
Experimental Validation
Simulation Studies
The PETL method was tested across three distinct simulation setups: population mean vectors, linear regression models, and nonparametric structural equation models. These experiments compared PETL against competing methods under both "Correctly Specified" (CM) and "Misspecified" (MS) conditions.
Key findings from these simulations include:
- Superior Accuracy: PETL achieved a lower Root Mean Square Error (RMSE) compared to standard mean estimators and thresholding methods.
- Oracle Performance: As sample sizes increased, PETL identified the true number of non-zero parameters with nearly 100% accuracy while maintaining minimal false discovery rates.
- Resilience to Correlation: PETL proved particularly effective in datasets with high variable correlation; its efficiency in identifying sparse structures actually improved relative to other methods as correlation increased.
Boston Housing Study
The authors applied PETL to the classic Boston Housing dataset (506 observations, 14 variables). By expanding the dataset to 92 moment conditions through the inclusion of interaction effects, they demonstrated PETL's utility in high-dimensional, real-world contexts. PETL successfully identified critical interaction effects—such as the combined impact of crime rates and property taxes—that exhibited higher statistical significance and smaller standard errors than those identified by Penalized Empirical Likelihood (PEL).
Discussion and Insights
The most significant contribution of this research is the demonstration of PETL's stability. While PEL’s bias grows significantly when a model is misspecified, PETL remains robust. Furthermore, the extension of Wilks’ Theorem to this framework is a major theoretical milestone, as it allows researchers to construct reliable confidence intervals and perform hypothesis tests using familiar tools, despite the complexities of unknown likelihoods and high dimensionality.
Limitations and Future Work
Despite its strengths, the authors identify several avenues for future research:
- Parameter-to-Moment Ratio: The current theory assumes the number of parameters is less than the number of moment conditions ($p < r$). Extending this to $p > r$ scenarios remains a challenge.
- Missing Data: The framework currently assumes completely observed datasets. Adapting PETL for missing data scenarios common in economics and medicine is a priority.
- Influence Analysis: Identifying outliers or influential observations that disproportionately affect the model in a growing-dimensional framework is an open area for investigation.
Summary updated