Efficient Estimation in NPIV Models: A Comparison of Various Neural Networks-Based Estimators

Jiafeng Chen, Xiaohong Chen, Elie TamerarXiv 2021 · OpenReview

TL;DR

This paper introduces two efficient methods for estimating average derivatives in complex economic models with unobserved factors, using neural networks as powerful function approximators. The study finds that while neural networks can perform well, tuning them is tricky, and simpler spline-based methods sometimes offer comparable or even better results in practice.

What Problem Does This Paper Address?

In economics, we often want to understand how a change in one variable affects an outcome, even when that variable is influenced by unobserved factors (endogeneity). This is particularly challenging when the relationships involved are complex and non-linear, and involve many variables. Traditional methods struggle with this "curse of dimensionality." This paper tackles this by using Artificial Neural Networks (ANNs) as "sieves" to approximate these complex, high-dimensional relationships in Nonparametric Instrumental Variables (NPIV) models. Specifically, they focus on estimating "weighted average derivatives" (WADs), which are crucial for understanding causal effects in these settings. The core problem is how to efficiently and reliably estimate these WADs when the underlying functions are unknown and potentially very complex, using ANNs.

Methods

The authors propose two main efficient estimation procedures for WADs using ANNs:

  1. Orthogonalized Plug-in with Optimally-Weighted Sieve Minimum Distance (OP-OSMD): This is a two-step procedure. First, it estimates the unknown NPIV function using an "optimally-weighted sieve minimum distance" approach with ANNs. Second, it uses this estimate to compute the WAD. This method is designed to be numerically stable and easier to tune.
  2. Sieve Efficient Score (ES): This procedure is based on the "efficient influence function," which is a theoretical construct that guides efficient estimation. It uses ANN sieves to approximate the components of this influence function and then estimates the WAD.

Both methods are designed to be root-n asymptotically normal, meaning they behave well with large sample sizes. The paper also provides a detailed "practitioner's recipe" for implementing these methods. For comparison, they also consider simpler, less efficient estimators (P-ISMD and IS) and use both ANN and spline-based sieves.

Datasets

The paper uses two empirical datasets:

  • Gasoline Demand Data: From the 2001 National Household Travel Survey, used to estimate price elasticity of gasoline demand.
  • Strawberry Demand Data: From the Nielsen dataset, used to estimate demand for strawberries.

These datasets are used to illustrate the application of their proposed methods to real-world economic problems. The paper also introduces several newly designed Monte Carlo simulation studies to rigorously evaluate the finite-sample performance of their estimators under various conditions.

Key Insights and Ideas

  • ANNs as powerful tools for complex economic models: The paper demonstrates that ANNs can effectively approximate the complex, high-dimensional functions found in NPIV models, which are relevant for many economic applications.
  • Tuning is critical (and difficult) for ANNs: A major takeaway is that achieving good performance with ANN estimators requires careful tuning of hyperparameters (like learning rate, network architecture, etc.). This tuning process can be delicate and non-convex optimization can lead to unstable results.
  • OP-OSMD is easier to tune than ES: The OP-OSMD procedure, while efficient, is found to be easier to implement and tune in practice compared to the ES procedure.
  • Splines can be competitive: Surprisingly, simpler spline-based estimators often perform as well as, or even better than, ANN estimators, especially when the underlying functions are relatively smooth. Splines also tend to be more stable and require less tuning.
  • The theory-practice gap: The authors note a persistent gap between theoretical expectations for ANN performance and their actual performance in finite samples, particularly concerning stable inference.

Key Results

  • ANNs can perform well: Given proper tuning, both ANN OP-OSMD and ANN ES estimators can achieve good finite-sample performance, even with up to 13 continuous covariates.
  • OP-OSMD generally preferred for ease of use: ANN OP-OSMD estimators are found to be easier to tune than ANN ES estimators.
  • Spline estimators are robust: Spline-based estimators (OP-OSMD, P-ISMD, IS, ES) are often less biased, more stable, and more accurate than their ANN counterparts, even in high-dimensional settings.
  • Stable inference is challenging with ANNs: Achieving stable statistical inferences (like confidence intervals) is more difficult with ANN-based estimators compared to spline-based ones.
  • Empirical applications show ANN NPIV's utility: The methods are applied to gasoline and strawberry demand data, estimating average price derivatives, demonstrating their practical applicability.

Limitations and Future Work

  • Tuning complexity: The paper highlights the significant challenge of tuning ANN estimators, which requires careful selection of numerous parameters. This is a major practical hurdle.
  • Stability of inference: Achieving stable inference with ANN estimators remains difficult, suggesting a need for further research in this area.
  • Splines' competitiveness: The finding that splines can outperform ANNs in some scenarios suggests that the choice of sieve is crucial and depends on the specific problem.
  • Bridging theory and practice: The authors explicitly state that more research is needed to close the gap between approximation theory for ANNs and their computational performance in nonparametric models with endogeneity.
  • Exploring nonlinear sieves for all components: While the paper uses ANNs for the main structural function, it notes that using nonlinear sieves for other nuisance parameters (like instrument projection or covariance functions) is computationally challenging and an area for future work.

Summary updated

Abstract

Artificial Neural Networks (ANNs) can be viewed as nonlinear sieves that can approximate complex functions of high dimensional variables more effectively than linear sieves. We investigate the performance of various ANNs in nonparametric instrumental variables (NPIV) models of moderately high dimensional covariates that are relevant to empirical economics. We present two efficient procedures for estimation and inference on a weighted average derivative (WAD): an orthogonalized plug-in with optimally-weighted sieve minimum distance (OP-OSMD) procedure and a sieve efficient score (ES) procedure. Both estimators for WAD use ANN sieves to approximate the unknown NPIV function and are root-n asymptotically normal and first-order equivalent. We provide a detailed practitioner's recipe for implementing both efficient procedures. We compare their finite-sample performances in various simulation designs that involve smooth NPIV function of up to 13 continuous covariates, different nonlinearities and covariate correlations. Some Monte Carlo findings include: 1) tuning and optimization are more delicate in ANN estimation; 2) given proper tuning, both ANN estimators with various architectures can perform well; 3) easier to tune ANN OP-OSMD estimators than ANN ES estimators; 4) stable inferences are more difficult to achieve with ANN (than spline) estimators; 5) there are gaps between current implementations and approximation theories. Finally, we apply ANN NPIV to estimate average partial derivatives in two empirical demand examples with multivariate covariates.
BibTeX
@misc{chen2021efficient,
  author = {Jiafeng Chen and Xiaohong Chen and Elie Tamer},
  title = {{Efficient Estimation in NPIV Models: A Comparison of Various Neural Networks-Based Estimators}},
  year = {2021},
  eprint = {2110.06763},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2110.06763},
}
Figure