From Stacked Predictions to Decisions: A Contextual Optimization Approach
Lists
Only you see this
Abstract
Optimization models and decision frameworks have seen increased use of the combination of multiple first-level learners using ensemble methods to improve predictive performance. In this work, we cast stacking as a contextual optimization problem, where a collection of $T$ first-level learners $\{h_t\}_{t=1}^{T}$ serve as context, a meta-learner estimates the target $Y$ conditional on this context, and decisions are selected to minimize the resulting conditional expected cost. We propose an empirical residual-based sample average approximation (ER-SAA) pipeline that incorporates the predictive uncertainty into the decision stage. Specifically, we (i) obtain out-of-sample first-level learners through cross-fitting, (ii) fit a meta-learner and estimate a possibly heteroskedastic scale map, and (iii) add back cross-fitted residuals as decision scenarios. This construction preserves stacking’s modeling flexibility and at the same time aligns learning with the conditional decision objective.
BibTeX
@inproceedings{guo2025from,
author = {Yanru Guo and Ruiwei Jiang and Siqian Shen},
title = {{From Stacked Predictions to Decisions: A Contextual Optimization Approach}},
booktitle = {MLxOR @ NeurIPS},
year = {2025},
url = {https://openreview.net/forum?id=vyJmWcDTa0},
}Loading related content...