Exponentially tilted likelihood inference on growing dimensional unconditional moment models

Niansheng Tang, Xiaodong Yan, Puying Zhao

TL;DR

This paper introduces Penalized Exponentially Tilted Likelihood (PETL), a robust framework for performing variable selection and parameter estimation in high-dimensional models where the true likelihood function is unknown. By combining exponential tilting with sparsity-inducing penalties, the authors provide a method that maintains "oracle" accuracy and remains reliable even when the underlying statistical model is misspecified or features are highly correlated.

What Problem Does This Paper Address?

In modern data science, we often deal with "growing-dimensional" data, where the number of parameters ($p$) increases alongside the sample size ($n$). A common hurdle in these scenarios is that we frequently lack a well-defined likelihood function. Instead, we rely on unconditional moment models—equations that describe the expected behavior of the data without assuming a full probability distribution.

While Empirical Likelihood (EL) is a popular tool for these models, it has a major weakness: it is highly sensitive to model misspecification. If your moment conditions are slightly "off," EL performance degrades rapidly. Furthermore, when variables are highly correlated or the number of parameters is large, traditional methods often suffer from slow convergence or fail to identify the truly important variables. This paper seeks to bridge this gap by providing a robust, high-dimensional inference tool that handles both "messy" models and complex variable dependencies.

Methods

The authors propose the Penalized Exponentially Tilted Likelihood (PETL). The core components of their approach include:

  1. Exponential Tilting (ET): Unlike Empirical Likelihood, which uses a multinomial approach, ET is based on minimizing the Kullback-Leibler divergence between empirical frequencies and weights. This makes it inherently more robust to misspecified models.
  2. SCAD Penalty: They incorporate the Smoothly Clipped Absolute Deviation (SCAD) penalty into the ET framework. This allows the model to automatically perform variable selection by shrinking the coefficients of unimportant variables to exactly zero.
  3. Sieve Method: To handle the "ill-posed" problem where the number of parameters diverges, they use a sieve approach to constrain the parameter space, ensuring the estimator remains consistent.
  4. Asymptotic Theory: The authors rigorously prove that their estimator possesses oracle properties—meaning it performs as well as if you knew the true set of important variables in advance. They also demonstrate that a version of Wilks’ Theorem holds, allowing researchers to use standard chi-squared tests for hypothesis testing.

Datasets

The paper validates the PETL method using both extensive simulations and a classic real-world dataset:

  • Simulated Experiments: Three distinct setups testing population mean vectors, linear regression models, and nonparametric structural equation models. These were designed to compare PETL against competitors under both "Correctly Specified" (CM) and "Misspecified" (MS) conditions.
  • Boston Housing Study: A dataset of 506 observations and 14 variables. The authors expanded this to 92 moment conditions by including interaction effects to demonstrate PETL's ability to handle high-dimensional, correlated real-world data.

Key Insights and Ideas

  • Robustness is Key: The most significant "aha moment" is the demonstration that PETL significantly outperforms Penalized Empirical Likelihood (PEL) when the model is misspecified. While PEL's bias grows when the model is wrong, PETL remains stable.
  • Handling Correlation: PETL is particularly effective for datasets with high correlation among variables. In simulations, as correlation increased, PETL's efficiency in identifying the correct sparse structure actually improved relative to other methods.
  • The "Wilks" Extension: Showing that the PETL ratio statistic follows a central chi-squared distribution is a major theoretical win. It means that despite the complexity of high dimensions and unknown likelihoods, researchers can still construct reliable confidence intervals and perform hypothesis tests using familiar tools.

Key Results

  • Superior Accuracy: In regression simulations, PETL achieved lower Root Mean Square Error (RMS) compared to standard mean estimators, hard-thresholding, and soft-thresholding methods.
  • Oracle Performance: PETL consistently identified the true number of non-zero parameters (e.g., 3 out of 19) with nearly 100% accuracy as the sample size grew, while maintaining minimal "false discovery" rates.
  • Boston Housing Insights: When applied to the Boston Housing data, PETL identified critical interaction effects (like the combined impact of crime rates and property tax) that were more statistically significant and had smaller standard errors than those found by PEL.

Limitations and Future Work

The authors acknowledge a few areas for further exploration:

  • Parameter-to-Moment Ratio: The current theory assumes $p < r$ (fewer parameters than moment conditions). Extending this to cases where $p > r$ remains a challenge.
  • Missing Data: The current framework assumes a "completely observed" dataset. In fields like economics or medicine, data is often missing, and adapting PETL for these scenarios is a stated goal for future research.
  • Influence Analysis: The paper notes that identifying "influential observations" (outliers that disproportionately affect the model) in a growing-dimensional moment framework is an open and important topic.

Summary updated

Abstract

Growing-dimensional data with likelihood unavailable are often encountered in various fields. This paper presents a penalized exponentially tilted likelihood (PETL) for variable selection and parameter estimation for growing dimensional unconditional moment models in the presence of correlation among variables and model misspecifica- tion. Under some regularity conditions, we investigate the consistent and oracle proper- ties of the PETL estimators of parameters, and show that the constrainedly PETL ratio statistic for testing contrast hypothesis asymptotically follows the central chi-squared distribution. Theoretical results reveal that the PETL approach is robust to model mis- specification. We also study high-order asymptotic properties of the proposed PETL estimators. Simulation studies are conducted to investigate the finite performance of the proposed methodologies. An example from the Boston Housing Study is illustrated.
BibTeX
@article{tang2017exponentially,
  author = {Niansheng Tang and Xiaodong Yan and Puying Zhao},
  title = {{Exponentially tilted likelihood inference on growing dimensional unconditional moment models}},
  journal = {Journal of Econometrics},
  year = {2017},
  doi = {10.1016/j.jeconom.2017.08.018},
}

Loading related content...

Figure