D-vine quantile regression with discrete variables

Niklas Schallhorn, Daniel Kraus, Thomas Nagler, Claudia CzadoarXiv 2017 · DOI · arXiv

Abstract

Quantile regression, the prediction of conditional quantiles, finds applications in various fields where understanding the full conditional distribution is more valuable than simply predicting the mean. However, many real-world datasets contain a mix of continuous and discrete variables, which poses a challenge for traditional copula-based methods. This paper proposes two novel quantile regression approaches that generalize continuous D-vine quantile regression to handle mixed discrete-continuous data. These methods provide flexible modeling, enable fast predictions, and automatically resolve issues like quantile crossing and complex covariate interactions. The first approach, Parametric D-vine Quantile Regression (PDVQR), modifies the recursive formulas to account for discreteness. The second, Nonparametric D-vine Quantile Regression (NPDVQR), utilizes continuous convolution to transform discrete variables. A simulation study and a real-world application predicting bike rentals demonstrate the superior predictive performance of these methods.

Introduction

Quantile regression is a powerful statistical tool used to predict specific points in a distribution, such as the median or extreme percentiles. Unlike standard linear regression, it provides a more comprehensive view of the relationship between variables. Despite its utility, traditional quantile regression faces significant hurdles: linear models often fail to capture non-linear interactions without manual feature engineering, and advanced copula methods typically assume data is continuous.

In datasets containing discrete variables (e.g., counts or categories), the underlying copula—the mathematical function that describes the dependence structure—is no longer uniquely defined. This paper bridges this gap by extending D-vine quantile regression to accommodate mixed-type datasets, ensuring that the flexibility of vine copulas can be applied to a broader range of real-world problems.

Methodology

The researchers introduce two distinct strategies for integrating discrete variables into the D-vine framework.

Parametric D-vine Quantile Regression (PDVQR)

This approach maintains the use of parametric copula families (such as Gaussian or Clayton). The authors modify the "h-functions"—the recursive building blocks of vine copulas—to account for the "jumps" present in discrete distribution functions. Because these modified functions are more difficult to invert analytically than their continuous counterparts, the model employs numerical inversion to determine the final predicted quantiles.

Nonparametric D-vine Quantile Regression (NPDVQR)

NPDVQR utilizes a technique known as continuous convolution. By adding a small amount of uniform noise (jittering) to discrete variables, the data is effectively made continuous. The model then performs nonparametric kernel density estimation on this transformed data. This approach preserves the mathematical simplicity of continuous vine operations while remaining statistically valid for discrete inputs once the results are related back to the original scale.

Both methods were benchmarked against Linear Quantile Regression (LQR), Boosted Additive Quantile Regression (BAQR), and standard Nonparametric Quantile Regression (NPQR).

Experimental Evaluation

Simulation Study

The methods were tested using simulated 3D and 5D datasets with varying signal-to-noise ratios and sample sizes ($n=250$ to $1000$). The simulations incorporated different degrees of "discreteness" using binomial distributions. The results indicated that in non-linear scenarios, D-vine methods—particularly NPDVQR—consistently achieved lower Mean Relative Absolute Scaled Error (MRASE) compared to linear benchmarks.

Real-World Application: Bike Rental Prediction

The authors applied their models to the UCI Bike Sharing Dataset to predict daily bike rental counts in Washington, D.C. This dataset is a classic example of mixed data, featuring continuous predictors (temperature, humidity) and discrete ones (weather situation, season).

NPDVQR and BAQR emerged as the top performers across all quantile levels (0.1, 0.5, and 0.9). Notably, NPDVQR successfully modeled a non-monotonic relationship between temperature and rentals: bike usage increased with temperature up to approximately 32°C (90°F) before declining as the heat became excessive. Standard linear models failed to capture this decline, highlighting the importance of the D-vine's flexible structure.

Key Insights and Advantages

  1. Automatic Interaction Handling: D-vines naturally capture complex dependencies and interactions between variables without requiring the researcher to manually specify interaction terms.
  2. Prevention of Quantile Crossing: A common failure in simple quantile regression is "quantile crossing," where a predicted higher quantile (e.g., 90th) is lower than a predicted lower quantile (e.g., 10th). The D-vine structure inherently prevents this by construction.
  3. Handling Non-Monotonicity: While parametric copulas often assume monotonic relationships, the nonparametric (NPDVQR) approach is highly effective at capturing "U-shaped" or other non-monotonic patterns.

Conclusion and Future Work

The proposed D-vine methods offer a robust framework for quantile regression with mixed data types. While PDVQR provides a rigorous parametric foundation, it is computationally intensive due to numerical inversions and limited by the shapes of existing copula families. NPDVQR offers greater flexibility and speed, particularly in high-dimensional or non-monotonic settings. Future research will focus on developing new parametric copula families capable of modeling non-monotonic dependencies to further enhance the robustness of the parametric approach.

Summary updated

Abstract

Quantile regression, the prediction of conditional quantiles, finds applications in various fields. Often, some or all of the variables are discrete. The authors propose two new quantile regression approaches to handle such mixed discrete-continuous data. Both of them generalize the continuous D-vine quantile regression, where the dependence between the response and the covariates is modeled by a parametric D-vine. D-vine quantile regression provides very flexible models, that enable accurate and fast predictions. Moreover, it automatically takes care of major issues of classical quantile regression, such as quantile crossing and interactions between the covariates. The first approach keeps the parametric estimation of the D-vines, but modifies the formulas to account for the discreteness. The second approach estimates the D-vine using continuous convolution to make the discrete variables continuous and then estimates the D-vine nonparametrically. A simulation study is presented examining for which scenarios the discrete-continuous D-vine quantile regression can provide superior prediction abilities. Lastly, the functionality of the two introduced methods is demonstrated by a real-world example predicting the number of bike rentals.
BibTeX
@misc{schallhorn2017d,
  author = {Niklas Schallhorn and Daniel Kraus and Thomas Nagler and Claudia Czado},
  title = {{D-vine quantile regression with discrete variables}},
  year = {2017},
  eprint = {1705.08310},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1705.08310},
}
Figure