Searching for What the Stacker Needs, Not What Scores Best Alone

Automated machine learning systems usually build a stacked ensemble in two separate steps. First, a search procedure explores a tree-shaped space of models and settings, meaning the settings that exist depend on which model family was picked: a support vector machine has a kernel width, a random forest has a depth. This search looks for configurations that score well on their own. Second, a meta-learner (a small model trained on the held-out predictions of the base models) combines whatever the search happened to find. This note argues that the first step is optimizing the wrong target. When the final predictor is a learned stacker, a candidate model is worth training only to the extent that adding it lowers the stacker's held-out loss. That marginal value is different from standalone accuracy and different from raw prediction diversity. It can be measured exactly for every model already trained, and it can be learned by a surrogate that respects the conditional structure of the search space.

The practical stake is sharpest when the ensemble has a size limit, which production latency and memory budgets impose. The AMLB benchmark documents how heavy top stacked systems are at inference. Under a cap of $K$ models, the question "which $K$ models should we train and tune?" is no longer answered by "the $K$ best individually." We propose a search procedure whose acquisition target is the stacker's marginal gain, with branch-aware surrogates. We test whether it produces smaller and better-calibrated stacks than both decoupled search-then-stack pipelines and diversity-driven search.

Core Claim

Let $P$ be the current pool of trained base configurations. Let $S(P)$ be a fixed-form, regularized stacker fit on their out-of-fold predictions, and let $L(\cdot)$ be the cross-validated log-loss of that stacker, estimated by nested folds over the out-of-fold matrix. The quantity the search should chase for a candidate configuration $c$ is

$$\Delta(c \mid P) = L\big(S(P)\big) - L\big(S(P \cup \{c\})\big).$$

For every evaluated configuration, $\Delta$ costs one refit of a small meta-learner on an $N \times |P|$ matrix, where $N$ is the number of training rows. That is negligible next to training a base model. Our claim has three parts. First, a surrogate trained to predict $\Delta$ allocates evaluations differently from one trained on standalone loss. It should shift budget away from the dominant family, typically gradient-boosted trees, much earlier. Second, at a fixed pool cap this yields lower test log-loss than the same pool size chosen by standalone score or by a diversity proxy. Third, because $L$ is a proper scoring rule applied to the combined prediction, the resulting stacks avoid the calibration degradation reported for accuracy-driven stacking.

The target is non-stationary: $\Delta$ for a family shrinks as that family fills the pool. This is exactly the kind of drift that MaxUCB flags as a problem for decomposed search, which treats each model family as a bandit arm and chases its peak individual reward. We handle the drift by conditioning the surrogate on a summary of the pool, rather than assuming a fixed reward per branch.

Relation to Prior Work

Four papers come close, and each leaves the gap open in a different way.

Two papers make the search ensemble-aware but stop at linear combination. Ensemble Optimization runs Bayesian optimization in which each candidate is scored by the loss of a fixed-size averaged ensemble with that candidate swapped into one slot. DivBO adds a second surrogate that predicts Euclidean distance between probability vectors, combined with greedy weighted ensemble selection. These papers motivate our work, since both show that ensemble-aware search beats ensembling the trajectory of ordinary search. Neither uses a learned stacker, neither handles a pool-size cap as the central constraint, and DivBO's diversity measure rewards models that are different even when they are wrong. We replace that proxy with the stacker's own held-out gain.

Two papers optimize the stack itself but not what feeds it. PSEO tunes stack depth, dropout, retention and blender choice over a fixed pre-computed pool. Autostacker evolves multi-layer topologies with default base hyperparameters. PSEO supplies our strongest downstream combiner, and we reuse it as a baseline stacker. Our contribution sits upstream: deciding which pool PSEO-style stacking receives.

AutoGluon-Tabular constrains the proposal. It shows that multi-layer stacking with skip connections over preset defaults beats extensive search, so any gain we claim must survive a comparison against simply training a broad default portfolio.

For the conditional structure we borrow existing methods rather than inventing a kernel. Candidates are the conditional kernel of Levesque et al., which zeroes covariance across branches, and tree-structured density estimation from Bergstra et al.. We rely on them because flattening with dummy values distorts distances (HPOBench). The richer shared encoders of AttnBO and the branching-and-nested GP are optional upgrades, not requirements.

Finally, the tabular foundation model ensembling study supplies two cautions. Stacking near-redundant models gains almost nothing, since those models show a pairwise Q-statistic of 0.961. And logistic-regression stacking gave the worst calibration. Our objective is chosen to confront the second point directly.

Research Program

The search space has two levels. The top level is the model family (gradient-boosted trees, random forests, extra-trees, k-nearest neighbors, MLPs, linear models, and optionally a tabular foundation model). Below each family sits that family's own hyperparameter subtree.

At each step the procedure does four things. It maintains the out-of-fold prediction matrix of the pool. It computes the realized $\Delta$ for the most recent addition. It updates a per-branch surrogate whose inputs are the configuration's active hyperparameters together with a short pool descriptor: the count of pool members per family, the current stacker loss, and pool size. It then proposes the next configuration by expected improvement in $\Delta$.

A family-level allocation rule chooses the branch using the upper confidence bound of predicted $\Delta$ within each branch, so a saturated family is abandoned even if it still holds the best standalone models. When the pool reaches the cap $K$, the procedure switches to swap proposals: a candidate is scored by $\Delta$ after removing the pool member with the smallest leave-one-out contribution.

The stacker $S$ is deliberately modest: multinomial logistic regression on log-probabilities, with L2 penalty and temperature both selected by inner cross-validation. It is held fixed during search so that $\Delta$ is well defined. After search, the final pool can be handed to any combiner, including PSEO. A separate outer test split is never touched during selection, because choosing models by $\Delta$ on the same folds is itself a source of optimism.

The analysis component asks when $\Delta$-targeting should help. Our hypothesis is that the gap is largest at small $K$ and on datasets where family rankings are close. It should vanish as $K$ grows large enough that any broad portfolio covers the useful directions.

Baselines

The naive reference points are the single best tuned model found by conditional-kernel Bayesian optimization under the full budget, and AutoGluon's default portfolio truncated to $K$ models by its own ensemble weights. The second one tests whether any search is needed at all.

The strongest existing competitor is a decoupled pipeline. Hierarchical CASH (conditional GP or tree-structured Parzen estimator) runs for the budget, the top-$K$ configurations by validation loss form the pool, and PSEO's tuned stacking is applied on top. PSEO is the right downstream half because it reports an average test rank of 2.96 against 15 baselines and a 10.4% average test-loss reduction over AutoGluon's best-quality mode on 80 OpenML datasets (PSEO).

The second strong competitor is DivBO's search followed by the same PSEO stacker. DivBO reports average test rank 1.73 across 15 OpenML classification tasks (DivBO). We also include Ensemble Optimization adapted to the cap, since it is the closest prior use of ensemble loss as the search target.

For fairness, every method draws from the same conditional space, receives the same number of base-model evaluations and the same wall-clock limit, uses the same folds, and ends with the same final combiner (both our fixed stacker and PSEO). Where the precomputed benchmark allows, all methods select from an identical finite set of already-evaluated configurations, so differences come from the selection policy alone.

Three ablations isolate the key ingredient. The first swaps the surrogate target from $\Delta$ to standalone loss, which reduces to CASH. The second swaps it to DivBO's diversity distance. The third keeps $\Delta$ but uses a flattened surrogate with dummy values. Beating the baselines means lower outer-test log-loss at $K \in \{4, 8, 16\}$, with a statistically significant average rank improvement over decoupled CASH plus PSEO and over DivBO plus PSEO, and no worse expected calibration error.

Why not just…?

Why not just train a broad default portfolio and let PSEO or AutoGluon stack it? This is the real threat. AutoGluon's own evidence is that breadth beats search, and PSEO already improves the stacking over whatever pool exists. If the precomputed portfolio is large and the pool is uncapped, we expect our advantage to disappear. The experiments settle this by sweeping $K$ and budget and reporting where the curves cross. If $\Delta$-targeting does not win at small caps, the idea fails.

Why not just run DivBO, since it already makes CASH ensemble-aware? DivBO's surrogate predicts distance between probability vectors, and a distant model can simply be a bad model. A learned stacker extracts value from a model's errors only when they are correlated with the target in a usable way. The diversity-target ablation compares the two targets under an otherwise identical search and combiner, so this is answered directly rather than argued.

Why not just use greedy ensemble selection and skip the learned stacker? The foundation-model study found greedy selection the most practical and best-calibrated option there. Its pool was homogeneous, though, and AutoGluon's evidence favors learned stacking on heterogeneous pools. We run greedy selection as an alternative final combiner on both our pools and the baseline pools. If greedy selection on a DivBO pool matches our full method, we will report that the learned stacker is unnecessary for this purpose.

Proposed Experiments

The primary testbed is TabRepo, with 7 model families across 200 datasets, as used for MaxUCB. It lets all policies be simulated over identical precomputed configurations. Whether its stored predictions include out-of-fold predictions suitable for stacking must be checked. If they do not, we regenerate them on a subset.

We confirm on PSEO's 80 OpenML datasets and on the AMLB suite under its 1-hour budget, with real training. No reported result was found on Lacuna for any method under a fixed ensemble-size cap, so the capped comparisons are new measurements rather than reproductions.

Metrics are outer-test log-loss, error, expected calibration error, average rank, anytime curves against evaluations, and inference time of the final stack.

We run three decisive checks. First, the realized $\Delta$ of chosen configurations should correlate more strongly with final test gain than their standalone rank does. We measure this with Kendall tau, comparable to DivBO's reported diversity-surrogate tau of up to 0.65. Second, the family-allocation traces should show earlier departure from the dominant family than under CASH. Third, adding a tabular foundation model branch alongside tree families should, under $\Delta$-targeting, raise the ensemble's gain over its best member beyond the +0.18% the foundation-model study reported for foundation-model-only cascades. That result would indicate whether a conditional heterogeneous space gets past that paper's diversity ceiling.