Vector quantile regression: An optimal transport approach

Guillaume Carlier, Victor Chernozhukov, Alfred Galichon

Introduction

Standard Quantile Regression (QR) has become a cornerstone of econometrics and statistics because it allows researchers to look beyond conditional means, offering a comprehensive view of how covariates influence the entire distribution of an outcome. However, classical QR is fundamentally limited to scalar (one-dimensional) outcomes. In many economic and social contexts, outcomes are inherently multivariate—such as a household's simultaneous expenditure on food, housing, and healthcare.

This paper introduces Vector Quantile Regression (VQR), a principled framework that extends quantile regression to multi-dimensional outcomes. By leveraging Optimal Transport (OT) theory, the authors define multivariate quantiles as gradients of convex functions, providing a robust way to model how a vector of dependent variables responds to covariates while maintaining a meaningful notion of "rank."

The Challenge of Multivariate Quantiles

The primary obstacle in moving to vector-valued outcomes ($Y \in \mathbb{R}^d$) is the absence of a natural, unique ordering in higher dimensions. In a one-dimensional setting, the 90th percentile is clearly defined. In two dimensions (e.g., food and housing expenditure), it is unclear what constitutes a "high" or "low" rank.

Previous attempts to define multivariate quantiles often lacked a clear regression framework, were computationally prohibitive, or failed to provide a "strong representation." A strong representation is a model where the outcome $Y$ can be expressed as a deterministic function of a latent rank $U$ and covariates $Z$. Without this, the interpretability of the model—specifically how an individual's latent propensity maps to their observed outcome—is lost.

Methodology: Optimal Transport and Brenier’s Theorem

The authors resolve the ordering problem by grounding VQR in Optimal Transport theory, specifically Brenier’s Theorem.

1. Monotonicity as a Gradient

In one dimension, a monotone function is the derivative of a convex function. The authors generalize this to higher dimensions: a multivariate quantile function is defined as the gradient of a convex potential function. This ensures that the mapping from a latent "rank" to an outcome is mathematically well-behaved and unique.

2. Conditional Vector Quantile Function (CVQF)

The CVQF is defined as the unique map that "pushes forward" a reference distribution (such as a uniform distribution on a unit cube) to the conditional distribution of the data. This provides a strong representation: $Y = Q_{Y|Z}(U, Z)$ almost surely, where $U$ represents a vector of latent ranks.

3. The Linear Model

The authors propose a linear specification for the CVQF: $$Y = \beta(U)^\top f(Z)$$ In this model, $\beta(U)$ is a matrix of coefficients that depends on the latent vector rank $U$, and $f(Z)$ represents a known set of transformations of the covariates. This structure allows the coefficients to have interpretations analogous to those in standard scalar quantile regression.

4. Optimization via Linear Programming

Finding these quantiles is shown to be equivalent to a covariance maximization problem, which is a dual form of the Monge-Kantorovich optimal transportation problem. In practice, the authors discretize the latent space, transforming the estimation into a large-scale Linear Programming (LP) problem that can be solved using standard optimization software.

Empirical Application: The Engel Dataset

To demonstrate the method, the authors apply VQR to the classic Engel Dataset (1857), which contains data on Belgian working-class household expenditures. This dataset is the "Hello World" of quantile regression, originally used by Koenker and Bassett (1878) to introduce the scalar version of the method.

In this application, the authors use Total Income as the covariate ($Z$) and a 2D vector of Food Expenditure and Housing/Fuel Expenditure as the outcomes ($Y$). The 2D VQR reveals insights that are invisible in separate 1D regressions. Specifically, the model identifies a significant negative cross-covariation between food and housing expenditures. At a fixed income level, an increase in the "rank" for food consumption often leads to a decrease in housing consumption, suggesting these two major budget items act as local substitutes for median-income households.

Key Insights and Results

  • Theoretical Consistency: The authors prove that when the outcome is a scalar ($d=1$), VQR reduces exactly to classical quantile regression.
  • The Vector Rank $U$: Instead of a single percentile, the rank $U$ is a vector. In the Engel example, $U_1$ can be interpreted as a "propensity to consume food" and $U_2$ as a "propensity to consume housing."
  • Deterministic Coupling: The model's strong representation means that for any given individual, their specific outcome is a direct result of their latent rank and observed characteristics.
  • Computational Feasibility: By using Kronecker products to vectorize the LP problem, the authors make it possible to compute multivariate quantiles for sample sizes typical in social science research.

Limitations and Future Directions

While VQR provides a rigorous foundation for multivariate analysis, it faces challenges in scaling. The number of variables in the LP optimization grows as the product of the sample size and the number of grid points in the latent space. For very high-dimensional outcomes ($d > 3$) or massive datasets, the computational burden remains significant.

Additionally, the results are sensitive to the discretization of the latent space $U$. Future research may focus on fully nonparametric extensions using "sieves" or more advanced grid selection techniques to handle higher-dimensional data more efficiently.

Summary updated

Abstract

We propose a notion of conditional vector quantile function and a vector quantile regression. A conditional vector quantile function (CVQF) of a random vector $Y$, taking values in $\mathbb{R}^{d}$ given covariates $Z=z$, taking values in $\mathbb{R}^{k}$, is a map $u\longmapsto Q_{Y|Z}(u,z)$, which is monotone, in the sense of being a gradient of a convex function, and such that given that vector $U$ follows a reference non-atomic distribution $F_{U}$, for instance uniform distribution on a unit cube in $\mathbb{R}^{d}$, the random vector $Q_{Y|Z}(U,z)$ has the distribution of $Y$ conditional on $Z=z$. Moreover, we have a strong representation, $Y=Q_{Y|Z}(U,Z)$ almost surely, for some version of $U$. The vector quantile regression (VQR) is a linear model for CVQF of $Y$ given $Z$. Under correct specification, the notion produces strong representation, $Y=\beta (U)^{\top}f(Z)$, for $f(Z)$ denoting a known set of transformations of $Z$, where $u\longmapsto\beta(u)^{\top}f(Z)$ is a monotone map, the gradient of a convex function and the quantile regression coefficients $u\longmapsto\beta(u)$ have the interpretations analogous to that of the standard scalar quantile regression. As $f(Z)$ becomes a richer class of transformations of $Z$, the model becomes nonparametric, as in series modelling. A key property of VQR is the embedding of the classical Monge–Kantorovich’s optimal transportation problem at its core as a special case. In the classical case, where $Y$ is scalar, VQR reduces to a version of the classical QR, and CVQF reduces to the scalar conditional quantile function. An application to multiple Engel curve estimation is considered.
BibTeX
@article{carlier2016vector,
  author = {Guillaume Carlier and Victor Chernozhukov and Alfred Galichon},
  title = {{Vector quantile regression: An optimal transport approach}},
  journal = {The Annals of Statistics},
  year = {2016},
  doi = {10.1214/15-aos1401},
}
Figure