Nonparametric augmented probability weighting with sparsity

Xin He, Xiaojun Mao, Zhonglei WangComput. Stat. Data Anal 2024 · OpenReview

TL;DR

This paper introduces a novel nonparametric augmented inverse probability weighting (AIPW) framework to address nonresponse in high-dimensional settings. By combining efficient kernel-based sparse learning for imputation with a penalized propensity score estimator, the method effectively handles a large number of covariates and achieves improved estimation efficiency and theoretical guarantees, including a central limit theorem.

What Problem Does This Paper Address?

Nonresponse is a common issue in data analysis, and ignoring it can lead to biased or inefficient results, especially when dealing with a large number of covariates (high-dimensional data). Traditional methods often struggle with high dimensionality, leading to issues like overfitting and numerical instability. While sparse learning techniques aim to identify relevant covariates, applying them to nonresponse problems in a nonparametric setting, particularly with theoretical guarantees, remains an open challenge. This paper tackles the problem of efficiently handling nonresponse in high-dimensional data by developing a nonparametric approach that leverages sparsity to select informative covariates.

Methods

The core of the proposed method lies in two main components:

  • Nonparametric Imputation with Sparse Learning: The authors employ an efficient kernel-based sparse learning algorithm to impute missing values. This algorithm uses a gradient-based approach within a reproducing kernel Hilbert space (RKHS) to identify truly informative covariates for estimating the conditional expectation of the response. This is a departure from traditional methods that might use all covariates, potentially leading to inefficiency. The method is inspired by recent work on kernel-based sparse learning but adapted to handle incomplete data.
  • Augmented Probability Weighting (AIPW) Framework: To enhance estimation efficiency and establish theoretical properties, an AIPW framework is adopted. This framework combines the nonparametric imputation with a penalized propensity score estimator. Specifically, a group lasso penalty is used to estimate the propensity scores, which helps in handling high-dimensional covariates for the response mechanism. The combination of these two components allows for a more robust and efficient estimator, with theoretical guarantees including a central limit theorem.

Datasets

  • Simulated Datasets: The paper uses two simulated regression models (linear and nonlinear) and two response models (logistic and multi-modal) to evaluate the performance of the proposed estimator. These simulations involve varying sample sizes (n) and numbers of covariates (p), including high-dimensional scenarios.
  • Supermarket Dataset: A real-world dataset from a supermarket in northern China, containing daily sales records and customer counts, is used for application. This dataset has a large number of covariates (p=6398) and a moderate sample size (n=464).

Key Insights and Ideas

  • Sparsity as a Key to High-Dimensional Nonresponse: The paper demonstrates that sparsity is crucial for effectively handling nonresponse in high-dimensional settings. By identifying and using only the truly informative covariates, the method avoids the pitfalls of using all covariates, such as overfitting and numerical instability.
  • Kernel-Based Sparse Learning for Imputation: The use of kernel-based sparse learning for nonparametric imputation is a novel approach that allows for flexible modeling of the relationship between covariates and the response without strong parametric assumptions.
  • Theoretical Guarantees in High Dimensions: A significant contribution is the establishment of theoretical properties, including asymptotic consistency and a central limit theorem, for the proposed AIPW estimator in a high-dimensional nonparametric setting. This is rare and valuable in machine learning literature.
  • Robustness to Response Model Specification: The numerical results suggest that the proposed AIPW estimator is robust, performing well even when the response model is misspecified, which is a common challenge in practice.

Key Results

  • Superior Performance in Simulations: The proposed AIPW estimator consistently outperformed competing methods (complete case, propensity score, deterministic imputation, naive AIPW) in terms of both Monte Carlo bias and standard error across various simulated scenarios, especially in high-dimensional settings.
  • Accurate Variance Estimation and Coverage: The proposed estimator's variance estimator showed low relative bias and achieved coverage rates close to the nominal 95% level, particularly for larger sample sizes.
  • Effective Real-Data Application: The method demonstrated superior performance on the supermarket dataset, showing a smaller standard error compared to other estimators, highlighting its practical utility.
  • Central Limit Theorem Established: The paper successfully establishes a central limit theorem for the AIPW estimator, which is a significant theoretical achievement for nonparametric methods in high dimensions.

Limitations and Future Work

  • Tuning Parameter Selection: While the paper mentions using a stability-based selection criterion for thresholding values and a ridge parameter, the sensitivity of the method to these tuning parameters might be a practical consideration.
  • Computational Cost: Although the method is described as computationally efficient, the kernel-based sparse learning and group lasso can still be computationally intensive, especially with very large datasets.
  • Future Work: The paper's theoretical framework and numerical results lay the groundwork for further research. Potential future directions could include exploring more advanced sparse learning techniques, investigating alternative propensity score estimation methods, and extending the framework to other types of missing data mechanisms.

Summary updated

Abstract

Highlights•A kernel-based nonparametric sparse learning algorithm is conducted for feature selection.•A nonparametric augmented inverse probability weighting framework is proposed.•Theoretical properties of the proposed estimators are investigated.•The proposed estimator outperforms its competitors numerically.
BibTeX
@inproceedings{he2024nonparametric,
  author = {Xin He and Xiaojun Mao and Zhonglei Wang},
  title = {{Nonparametric augmented probability weighting with sparsity}},
  booktitle = {Comput. Stat. Data Anal},
  year = {2024},
  url = {https://openreview.net/forum?id=3DJtNHobX3},
}
Figure