Abstract
This paper introduces a novel framework for estimating the entire conditional distribution of a response variable by modeling the quantile function as a mixture of basis functions. By leveraging specific spline properties and convex optimization, the method provides a flexible, interpretable model that inherently prevents "quantile crossing" and is mathematically equivalent to minimizing the Continuous Ranked Probability Score (CRPS). Empirical evaluations demonstrate that the proposed Factor Model of Mixture Quantiles (FMMQ) offers superior coverage reliability and computational efficiency compared to existing distributional regression techniques.
Introduction
Standard regression models typically focus on predicting the conditional mean of a response variable. However, in high-stakes fields such as finance, meteorology, and healthcare, understanding the full conditional distribution is essential for risk assessment and uncertainty quantification. Existing approaches to distributional regression often struggle with "quantile crossing"—where lower quantiles are numerically estimated to be higher than upper quantiles—and the "curse of dimensionality" as feature sets grow. Furthermore, many flexible models rely on non-convex optimization, leading to computational inefficiencies and susceptibility to local optima.
Methodology
The authors propose the Factor Model of Mixture Quantiles (FMMQ) to address these challenges. The core innovation lies in representing the conditional quantile function as a weighted sum:
$$G(p, x, a) = \sum f_i(x) Q_i(p)$$
where $Q_i(p)$ are basis quantile functions and $f_i(x)$ are weight functions dependent on input factors.
Structural Integrity and Optimization
To ensure valid distributions, the model utilizes I-splines for the quantile bases, which are inherently non-decreasing, and B-splines with non-negative coefficients for the weights. This structural choice mathematically guarantees that quantiles will not cross, removing the need for complex constrained optimization.
The authors demonstrate that minimizing the integrated pinball loss across all probability levels is equivalent to minimizing the Continuous Ranked Probability Score (CRPS). Since the model is linear with respect to its parameters, estimation remains a convex optimization problem, ensuring global convergence.
Scalability via Tensor Decomposition
To maintain efficiency in high-dimensional settings, the framework incorporates Canonical Polyadic (CP) tensor decomposition. This allows the parameter space to scale linearly with the number of dimensions. While the reduced-rank version uses an alternating optimization algorithm, it demonstrates rapid convergence in practice, typically reaching stability in fewer than 25 iterations.
Experimental Evaluation
The FMMQ model was evaluated using the UCI Combined Cycle Power Plant dataset and Monthly Consumer Price Index (CPI) data.
Performance and Reliability
On the Power Plant benchmark, FMMQ achieved competitive CRPS values and demonstrated superior coverage rates compared to Gaussian Mixture Regression (GMR) and Generalized Additive Models (GAMLSS). As shown in the comparison below, the FMMQ model maintains empirical coverage very close to the nominal levels, outperforming competitors in average difference (0.007 vs. 0.013 for GMR/GAM).
Computational Efficiency
FMMQ proved significantly faster to train than alternative methods. For the Power Plant dataset, FMMQ required only 11.1 seconds, whereas GMR and GAMLSS took 109 seconds and 29 seconds, respectively. This efficiency makes the model suitable for large-scale datasets where traditional distributional models might be computationally prohibitive.
Conclusion
The FMMQ framework provides a robust, efficient, and interpretable solution for nonparametric distributional regression. By embedding monotonicity into the model architecture and optimizing for CRPS via convex methods, it avoids common pitfalls like quantile crossing. Future research may explore automated hyperparameter selection for spline knots and the integration of heavy-tailed distributions to improve modeling of extreme risks.
Summary updated