Real-Patient Exemplars as the Meta-Learner in Stacked Diagnostic Ensembles

Stacking is a way to combine several different predictive models. Each "base" model makes a prediction, and a second "meta" model learns which of them to trust for which kind of case. In small medical cohorts it is often the most accurate approach available. It is also doubly opaque. The final decision is a nonlinear function of other models' probabilities, so even a faithful attribution method only says "the T2-weighted pancreas-head radiomics model mattered," never why. This matters most in fine-grained, ordinal diagnosis, where the task is to place a patient on a graded scale such as no-risk, low-risk or high-risk for a pancreatic cyst, or the five thyroid risk tiers. There, a clinician must decide whether a borderline case belongs one grade up or down.

We propose replacing the meta-model with a small set of real training patients. For a new patient, the meta-model states which of roughly twenty curated, histology-confirmed past patients the ensemble's behaviour resembles, in what proportions, and on which named base models. The prediction is then the mix of those patients' grades. The exemplars are chosen once, by a label-aware version of CUR decomposition. CUR approximates a matrix using a few of its actual rows and columns, rather than synthetic axes as singular value decomposition (SVD) does. Here the rows are patients and the columns are base models. The claim is that this case-based meta-model can match the accuracy of regularized tree-based stacking on ordinal grading, make fewer errors that skip a grade, and give an explanation a clinician can check against real cases.

Core Claim

Let $Z \in \mathbb{R}^{N \times MC}$ be the out-of-fold meta-feature matrix: for each of $N$ training patients, the log-probabilities that each of $M$ base models assigns to each of $C$ ordinal grades. Out-of-fold means each model's predictions come from folds it was not trained on. We select a set $I$ of $r$ exemplar patients (rows) and a set $J$ of base models (column groups, with all $C$ columns of one model taken together). A new patient with meta-feature vector $z$ is then represented as a convex mix of exemplars restricted to the chosen base models:

$$ w(z) = \arg\min_{w \ge 0,\ \mathbf{1}^\top w = 1} \Big\| z_J - \sum_{i \in I} w_i Z_{i,J} \Big\|_2^2, \qquad \hat p(y = c \mid z) = \sum_{i \in I} w_i(z)\, \mathbb{1}[y_i = c]. $$

Here $w_i$ is the weight on exemplar $i$ and $y_i$ is that exemplar's confirmed grade. The weights are sparse and nonnegative by construction. As a result, the explanation ("this cyst's ensemble profile is 0.6 × patient 37, high-risk at resection, and 0.4 × patient 112, low-risk on follow-up, as judged by the T2W head and body models") is the prediction itself, not a post-hoc account of it.

What is new is not "apply CUR to stacking" in the abstract. It is two specific changes to how CUR chooses rows. First, selection is label-aware. We run row volume sampling on $[Z, \lambda E(Y)]$, where $E(Y)$ embeds each ordinal grade as a cumulative indicator vector so that adjacent grades lie closer together, and $\lambda$ controls how much the labels count. Second, selection is quota-stratified: each grade, especially the rare high-risk grade, receives a guaranteed number of exemplars. Volume sampling favours subsets that are far from linearly dependent. Our hypothesis is that it therefore picks boundary cases between adjacent grades rather than the typical centre of each grade. Boundary cases are where ordinal mistakes happen.

Positioning Against Prior Work

The accuracy target and the motivating gap both come from Gaussian Meta-Space Augmentation for Stacking Ensembles in Multimodal IPMN Risk Stratification. That paper stacks 16 anatomical MRI streams for 3-grade pancreatic cyst risk. To keep tree meta-learners from overfitting, it adds synthetic meta-features sampled from class-conditional Gaussians fitted in log-probability space, a method it calls cUPMI. Its meta-learner offers no link to cases or features. The authors also flag that the high-risk class (about 142–155 of 678 patients) is too small for stable covariance estimates. Our meta-model estimates no covariance. It stores real exemplars whose meta-features are honest out-of-fold predictions, so it removes the component the authors identify as fragile rather than trying to stabilize it.

Two related threads supply the method. On the matrix side, Efficient volume sampling for row/column subset selection gives polynomial-time exact and approximate volume sampling with a $(k+1)$ bound on expected squared Frobenius error, which is the best possible. We reuse this sampler for rows. Fast Deterministic CUR Matrix Decomposition with Accuracy Assurance casts CUR as selecting whole groups of columns at once and adds provably exact safe screening. We reuse that group structure so that a base model enters or leaves with all its grade columns together. On the exemplar side, EBEK: Exemplar-Based Kernel Preserving Embedding shows that greedily chosen real data points can preserve a kernel's similarity structure and yield recognizable bases, for example individual faces instead of blurry eigenfaces. We take the same stance, applied to patients in the meta-space and used for supervised prediction rather than retrieval.

Several works constrain the design. Federated Data and Feature Selection by Generalized CUR Decomposition calls CUR explicitly "label-blind," and the review of Unsupervised Feature Selection for the $k$-means Clustering Problem makes the same point about leverage-score selection. High-variance directions need not follow grade boundaries. This is why we inject labels into the selection and test label-blind selection as an ablation. Tensor-CUR decompositions for tensor-based data is the closest point where CUR met diagnosis: selecting 8–16 of 128 spectral bands kept 3-class colon-tissue accuracy intact or better. It was a single-model, unsupervised, coarse-label pipeline, and it selected features, not patients. Ensembling Tabular Foundation Models - A Diversity Ceiling And A Calibration Trap warns that ensembles of strong models produce correlated, miscalibrated errors. We read this as a reason to prune redundant base models, and we measure calibration directly.

The granularity literature sets the evaluation. Soft multi-expert labels help on ambiguous boundaries, as in Pathologist-like explainable AI for interpretable Gleason grading in prostate cancer. Fine-grained thyroid tiers are hard even for strong image backbones, as in CMCNet: Aligning Ultrasound Image Embeddings with Textual TI-RADS Representations for Fine-Grained Thyroid Classification. Both support judging success by grade-skipping errors and boundary behaviour, not only by headline accuracy.

Research Program

The pipeline is built inside nested cross-validation. Within each outer training split, we train the heterogeneous base models with an inner out-of-fold protocol to produce an honest $Z$. On Cyst-X these are random forests on radiomics and ResNet-18 models for each of the four anatomical regions and two MRI sequences, mirroring the stacking paper. Rows and column groups are then selected from this $Z$ only, and outer-fold patients never become exemplars.

Column groups are chosen by grouped selection, with the reconstruction target extended by the label embedding. Row exemplars are chosen by approximate volume sampling on the label-augmented matrix, subject to per-grade quotas, and derandomized so that the set is reproducible.

The analytical question is whether label augmentation with a finite $\lambda$ trades a controlled amount of reconstruction error for boundary coverage. We will measure, as a function of $r$ and $\lambda$, both the relative reconstruction error of $Z$ and the fraction of exemplars lying within one grade of a decision boundary.

The interpretive deliverable is a fixed exemplar atlas per outer fold. Each exemplar is linked to its images, histopathology or follow-up outcome, and the named base models in $J$. We will measure how stable the atlas is across folds and seeds.

Baselines

The naive reference points are the single best base model chosen on inner validation, the unweighted average of all base models' probabilities, and a majority-grade predictor. Any meta-model that fails to beat the plain average is not worth its interpretive cost.

The strongest existing competitor on Cyst-X is the stacked random forest of Gaussian Meta-Space Augmentation for Stacking Ensembles in Multimodal IPMN Risk Stratification. It reports a binary AUC of 0.839 and a quadratic weighted kappa (QWK, an agreement score that penalizes errors more the further apart the grades are) of 0.595. That paper also reports that cUPMI adds 0.022 QWK to an XGBoost meta-learner in the radiomics-only setting. We will re-run both the stacked random forest and cUPMI-XGBoost on our exact outer folds and base-model outputs rather than rely only on the reported numbers, because our fold splits will differ. Every meta-model gets the same $Z$, the same seeds and the same budget of 50 hyperparameter configurations tuned on inner folds.

On the thyroid STN dataset, the reported reference is CMCNet with an ultrasound foundation-model backbone: about 75.17% balanced accuracy and 91.78% weighted F1 on the 5-tier task, from CMCNet. For our purposes CMCNet serves both as a baseline and as one base model in the pool.

We also run linear stacking with ordinal logistic regression, and sparse nonnegative stacking with a group lasso over base models. The latter is the supervised way to select columns.

Four ablations isolate the key ingredient. Label-blind CUR (with $\lambda=0$) tests whether supervision in the selection matters. Removing the quotas tests minority coverage. Random exemplars at matched $r$ tests whether selection matters at all. Per-grade k-medoids exemplars in meta-space tests whether volume-driven boundary coverage beats "typical case" selection.

Beating the strongest method means one of two things. The first is QWK within 0.02 of our re-run cUPMI-XGBoost, with $r \le 30$ exemplars, together with a lower rate of errors that skip a grade. The second is strictly higher QWK. Matching accuracy without the lower skip-grade rate would count only as an interpretability result.

Why not just…?

Why not just use a sparse linear stacker such as the group lasso, which already selects base models with supervision? For the column half of the problem this is a real threat, and we treat it as one. If group-lasso column selection matches our label-augmented column selection, we will adopt it. The contribution rests on the rows. A linear stacker explains through weights on base models, not through cases. Because it is linear in log-probabilities, it cannot express "trust the head model only when the body model disagrees," which is the kind of nonlinear interaction the IPMN paper cites as the reason to use trees. The exemplar mix is locally nonlinear. The head-to-head QWK comparison on identical $Z$ settles whether that matters.

Why not just use k-nearest neighbours over all training patients in meta-space? Nearest neighbours is case-based, but each query can retrieve a different set of neighbours from all 678 patients. That leaves no small atlas a clinician can vet once, and no guarantee that high-risk cases are represented. We will run it with tuned $k$. If it wins on accuracy while our atlas wins on stability and review cost, we will report the trade-off honestly.

Why not just pick class medoids as exemplars? This is the simplest competitor and a plausible one. Medoids choose central, typical cases. Our hypothesis is that ordinal errors arise at boundaries between adjacent grades, where medoids are sparse. The k-medoids ablation at matched $r$, scored on QWK and on accuracy among patients whose base models disagree most, tests this directly. If medoids match, the finding is that a case-based meta-model suffices and volume sampling is unnecessary, and we will say so.

Why not just apply SHAP to the XGBoost meta-learner and then to each base model? This yields importance scores on probabilities and features, with no guarantee that they reproduce the decision. We will run a deletion test on both methods. For ours, we remove the top-weighted exemplar and re-predict. For the post-hoc chain, we remove the top-ranked base model and re-predict. We then compare how much each prediction changes, as a measure of how faithful each explanation is.

Proposed Experiments and Decisive Checks

The primary benchmark is Cyst-X: 678 patients from three centres, 3 ordinal grades, 16 streams. Access to Cyst-X is a practical risk, and if it cannot be obtained, STN becomes the primary benchmark. We will also run a leave-one-centre-out split to test whether the exemplars transfer across hospitals.

The secondary benchmark is STN (600 nodules, 5 TI-RADS tiers). Its base models will be per-view image models built on the ultrasound foundation-model and BiomedCLIP backbones, plus CMCNet. As a scale check we use EyePACS diabetic-retinopathy grading, where the Lesion-Aware Transformer reports QWK 0.884 (Lesion-Aware Transformers for Diabetic Retinopathy Grading). There we expect exemplars to matter less, and we test whether $r$ must grow with $N$.

The metrics are QWK, binary high-risk AUC, balanced accuracy, the rate of errors that skip a grade, expected calibration error, how stable the atlas is across folds (Jaccard overlap between exemplar sets), and the drop in prediction under the deletion test.

We regard the proposal as confirmed if three results hold on Cyst-X. First, label-aware, quota-stratified CUR with at most 30 exemplars comes within 0.02 QWK of re-run cUPMI-XGBoost. Second, it beats both label-blind CUR and k-medoids at matched $r$ on boundary patients. Third, its high-risk AUC stays stable across seeds where cUPMI's varies. The proposal fails cleanly if k-medoids or tuned nearest neighbours match it on every metric, or if accuracy collapses below plain averaging at interpretable exemplar counts.