Conditional Diffusion for Nonparametric Instrumental Variable Quantile Regression

Xingdong Feng, Xinhong Jiang, Yuling Jiao, Lican Kang, Junwei LiuarXiv 2026 · arXiv

TL;DR

This paper introduces Deep Instrumental Variable Quantile Regression (DIVQR), a novel two-stage framework that combines conditional diffusion models with deep neural networks to tackle endogeneity in high-dimensional settings. By learning the full conditional distribution of data rather than relying on rigid, low-dimensional approximations, DIVQR overcomes the curse of dimensionality and provides strong theoretical guarantees even for heavy-tailed data.

What Problem Does This Paper Address?

In causal inference, endogeneity is a pervasive challenge where an observed explanatory variable (covariate) is correlated with the unobserved error term, often due to omitted variables or measurement errors. Standard regression models fail to identify true structural relationships under endogeneity. The classical remedy is the Instrumental Variables (IV) method, which uses an exogenous "instrument" that affects the outcome only through the endogenous covariate.

While classical IV methods target the conditional mean, Instrumental Variable Quantile Regression (IVQR) maps out the full conditional distribution, capturing heterogeneous effects across different quantiles (e.g., how an intervention impacts low-income vs. high-income groups differently). However, existing nonparametric IVQR methods rely on kernel-based or fixed-basis sieve approximations that break down in high-dimensional settings. Furthermore, applying deep learning directly to IVQR is hindered by a major technical obstacle: the underlying moment equations rely on a discontinuous indicator function, which makes gradient-based optimization notoriously difficult.

Methods

The authors propose DIVQR, a two-stage deep nonparametric estimator that bypasses traditional low-dimensional approximations:

  1. Stage I (Conditional Diffusion Learning): Instead of trying to invert the complex conditional moment operator directly, the model learns the joint conditional density of the outcome and endogenous covariates given the instrument ($p_{x,y|z}$). It achieves this using a variance-preserving conditional diffusion model trained via denoising score matching over a deep ReLU network. This generative model is then used to draw clean Monte Carlo samples.
  2. Stage II (Smoothed Empirical Risk Minimization): To overcome the non-differentiability of the quantile objective, the authors apply a kernel-smoothing technique to the indicator function. Using the samples generated in Stage I and an independent second dataset (sample-splitting), they approximate the conditional moment operator and estimate the structural quantile function by minimizing a smoothed empirical risk objective over a class of Deep Neural Networks (DNNs).

The authors also provide a unified non-asymptotic theoretical framework, establishing excess-risk bounds under unbounded support and heavy-tailed (polynomial-tail) data distributions.

Datasets

The method was evaluated using both synthetic and real-world data:

  • Simulation Studies: A complex, 10-dimensional non-linear structural regression function tested under both Gaussian and heavy-tailed ($t_3$) error distributions.
  • Real-World Application: The NBER–CES Manufacturing Industry Database (1958–2018), containing 26,339 observations. The goal was to model U.S. manufacturing output (log real value-added) as a structural quantile function of three endogenous inputs: labor, capital, and energy. A five-dimensional continuous instrument vector was constructed from historical input cost shares and national price indices to correct for productivity shock endogeneity.

Key Insights and Ideas

  • Generative Modeling for IV: Instead of directly regularizing operator inversion (the classical approach), DIVQR reframes the problem by leveraging the high-dimensional generative power of diffusion models to learn the underlying conditional data distribution itself.
  • Bridging Heavy and Light Tails: Traditional diffusion guarantees assume compactly supported (bounded) data. This paper develops an end-to-end total variation convergence guarantee under a polynomial-tail envelope. Remarkably, as the tail index grows, the theory continuously transitions to recover classical light-tailed minimax-optimal rates.
  • Decoupled Randomness: By utilizing a sample-splitting protocol between Stage I (diffusion training) and Stage II (moment optimization), the framework statistically isolates generative errors from function estimation errors, ensuring stable convergence.

Key Results

  • Simulation Performance: In the challenging heavy-tailed $t_3$ simulation ($d=p=10$), DIVQR demonstrated a clear trend of substantial test Mean Squared Error (MSE) reduction as sample sizes scaled from 1,000 to 3,000. For the median quantile ($\tau = 0.50$), the test MSE dropped from 0.0694 to 0.0111.
  • Real-Data Benchmarks: DIVQR was benchmarked against three classical nonparametric IVQR methods (Polynomial-Sieve, Nadaraya-Watson, and Kernel-Ridge). DIVQR achieved a strictly lower out-of-sample pinball loss across all target quantiles ($\tau \in \{0.25, 0.50, 0.75\}$), yielding a 22.5% to 36.1% reduction in loss compared to the best-performing baseline.
Method $\mathcal{P}_\tau$ at $\tau = 0.25$ $\mathcal{P}_\tau$ at $\tau = 0.50$ $\mathcal{P}_\tau$ at $\tau = 0.75$
Poly-Sieve IVQR 0.338 0.471 0.393
Nadaraya-Watson IVQR 0.372 0.493 0.330
Kernel-Ridge IVQR 0.346 0.490 0.388
DIVQR (Ours) 0.262 0.301 0.251

Limitations and Future Work

The authors note several avenues for future research:

  • Statistical Inference: The current work focuses on estimation and risk bounds; future work is needed to develop formal hypothesis testing and confidence intervals for the structural quantile function using cross-fitting and Neyman orthogonalization.
  • Non-Crossing Constraints: When estimating multiple quantiles simultaneously, curves can sometimes cross, which is logically inconsistent. Extending the framework to model the entire quantile process under non-crossing constraints remains an open challenge.
  • Adaptive Hyperparameters: The current theory relies on knowing the smoothness index and tail behavior of the conditional density to set optimal parameters. Developing fully data-driven, adaptive tuning procedures is a next step.

Summary updated

Abstract

This work proposes deep nonparametric Instrumental variable quantile regression (IVQR), a two-stage estimator that combines conditional diffusion modeling with a kernel-smoothed conditional moment formulation. In the first stage, we estimate the joint conditional distribution of the outcome and endogenous covariates given the instrument using a variance-preserving conditional diffusion model. In the second stage, we approximate the conditional moment operator through Monte Carlo sampling and a kernel-smoothed surrogate for the indicator function, and then estimate the structural quantile function by empirical risk minimization over deep neural networks. We establish an excess-risk bound for the proposed estimator and derive end-to-end total variation guarantees for the conditional diffusion model under unbounded support, explicitly accounting for score estimation, early stopping, and discretization errors. Our theory is developed under a polynomial-tail envelope on the data distribution and degenerates continuously to the exponential setting: as the tail index grows, the obtained excess-risk rate converges to the minimax-optimal rate of nonparametric regression, thus our heavy-tailed theory covers the classical light-tailed nonparametric guarantees as a limiting case. Simulation studies and a real-data application demonstrate that the proposed method outperforms existing nonparametric IVQR approaches, with gains that become increasingly pronounced as the dimensionality of the covariates and instruments increases.
BibTeX
@misc{feng2026conditional,
  author = {Xingdong Feng and Xinhong Jiang and Yuling Jiao and Lican Kang and Junwei Liu},
  title = {{Conditional Diffusion for Nonparametric Instrumental Variable Quantile Regression}},
  year = {2026},
  eprint = {2608.08204},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2608.08204},
}
Figure