Learning When to Ask for a Slide Label versus a Tumor Outline

Pathologists can supply two very different kinds of supervision for a whole slide image (a gigapixel scan of a tissue section). The first is a label for the whole slide, such as "metastasis present," which takes seconds. The second is an outline of where the tumor actually is, which can take many minutes. Models that learn from slide labels alone, known as attention-based multiple instance learning (MIL) models, treat each slide as a "bag" of thousands of small patches and learn a weighting over those patches. They can classify slides well and still attend to the wrong tissue. On CAMELYON16, a weakly supervised model reaches slide AUC 0.895 but localization Dice of only 0.215, and outlines on a random 10% of slides raise these to 0.924 and 0.350 (Attention-based Multiple Instance Learning with Mixed Supervision on the Camelyon16 Dataset). The practical question is therefore not only which slides to annotate. It is which kind of annotation to buy next, and for which slide, given how much budget remains.

We propose to train an acquisition policy with reinforcement learning (an agent that learns a decision rule from trial-and-error episodes and a reward signal) to make exactly this choice. At each step it either buys a cheap slide label for an unlabeled slide or buys an expensive region outline for a slide already known to be positive. Its reward is the improvement of the MIL model on held-out slide classification and localization. The policy is trained in simulation on fully annotated cohorts and then deployed frozen on a new cohort. The claim is testable in a simple way: under a fixed annotation-cost budget, the learned policy should beat the best hand-tuned rule for mixing the two annotation types across a range of cost ratios. It should also do so on a hospital cohort it never saw during training.

Core Claim

The value of a region outline depends on the model's state in a way that fixed rules cannot capture. Early on, when few slides are labeled, attention is diffuse and outlines are wasted. Later, when the model has latched onto a shortcut such as stroma or a staining artifact, one outline on the right slide can redirect the attention weights. A policy that sees how the attention is behaving, and not only how uncertain the prediction is, should learn when this switch pays off. It should also learn to target outlines at slides where attention is confidently wrong rather than merely uncertain. What is new is a learned policy over a mixed action space of bag labels and instance-level outlines for deep attention MIL, with a cost-aware budget. No paper in our search does this.

Where This Sits

Several lines of work each cover part of the problem. Heuristic multiple-instance active learning splits along the same axis we want to learn. Multiple-Instance Active Learning queries instance labels only inside bags already known to be positive, using an uncertainty-times-influence score on a logistic model. Its authors explicitly note that a complete system would decide when to query a new bag versus drill into an existing one. Bag-Level Aggregation for Multiple-Instance Active Learning queries whole bags only and finds that no single scoring rule wins across datasets. These two papers motivate the proposal: one frames the granularity decision as open, and the other shows fixed rules are brittle. Neither uses deep attention models or learned policies.

In pathology, Active Learning Enhances Classification of Histopathology Whole Slide Images with Attention-based MIL is the closest system. It uses Monte Carlo dropout (running the network several times with random units switched off to estimate uncertainty) to score slides by prediction and attention uncertainty. It then requests region masks for the chosen slides and trains with a loss that pulls attention toward the mask. We reuse its attention-guiding loss and its attention-uncertainty signal as state features. We differ in three ways: the query order is learned rather than ranked, the choice between slide labels and outlines is a decision rather than a fixed two-stage pipeline, and cost enters the objective. The mixed-supervision CLAM study supplies the evidence that outlines matter. Its outlines were assigned randomly, and we ask whether choosing them helps.

On the reinforcement learning side, Reinforced active learning for image segmentation trains a Q-network (a network that estimates the long-run value of each action) to pick image regions for dense labeling. It is rewarded by segmentation gain and reports roughly 30% less labeled data than the BALD uncertainty method on Cityscapes. Pre-Training Acquisition Functions by Deep Reinforcement Learning trains a policy on synthetic tasks and transfers it to real tabular data, with budget left as a state variable. These papers supply our method: a budget-aware state, reward from validation gain, and simulation-trained transfer. We carry these ideas into a two-level bag/instance setting that neither paper touches.

Finally, Dynamic Policy-Driven Adaptive Multi-Instance Learning (PAMIL) also combines reinforcement learning with MIL. Its agent picks which patches of an already-labeled slide to process at inference. It never queries an annotator, so it addresses a different problem. We mention it only to mark that boundary.

Research Program

The downstream learner is gated attention MIL (Attention-based Deep Multiple Instance Learning) on frozen patch features. When an outline is available, we add the attention-guiding loss from Sadafi et al. Freezing the features is deliberate. Retraining the aggregator takes minutes, which makes the reward loop affordable and avoids the computational bottleneck that limits reinforcement-learning active learning. Following DTFD-MIL, we use pseudo-bags (random subsets of a slide's patches treated as separate bags) during early rounds. This mitigates the diffuse-attention, small-cohort instability expected when only five to ten slides are labeled.

The policy scores candidates one at a time with a shared network, so the action space does not grow with pool size. Each candidate is either an unlabeled slide (action: buy a label at cost $c_b$) or a labeled positive slide without an outline (action: buy an outline at cost $c_r$). Each candidate is described by a small feature vector:

  • slide prediction probability and its Monte Carlo dropout variance;
  • entropy and dropout variance of the attention distribution;
  • overlap between the top-attended patches across dropout samples;
  • distance of the slide's attended embedding to already-labeled slides.

Global state adds the remaining budget fraction, counts of labels and outlines bought so far, and the current validation AUC. The per-step reward is

$$r_t = \frac{\left(\mathrm{AUC}_{t+1} - \mathrm{AUC}_t\right) + \lambda\left(\mathrm{Dice}_{t+1} - \mathrm{Dice}_t\right)}{c(a_t)}$$

where AUC is slide-level area under the ROC curve on a held-out validation split, Dice is localization overlap between thresholded attention and ground-truth tumor on validation positives, $\lambda$ sets how much localization matters, and $c(a_t)$ is the cost of the chosen action. Dividing by cost makes an outline earn its price. We train with a policy-gradient method over many simulated episodes. Each episode draws a random cost ratio $c_r/c_b$ and a random budget, so a single policy covers the range. Training cohorts are cross-validation folds of CAMELYON16, which has tumor outlines, plus MNIST-bags, where the "outline" is the identity of the positive digit. All simulated labels and outlines are revealed from existing ground truth.

Baselines

The naive reference is random acquisition with a fixed split of budget between labels and outlines, taken from the mixed-supervision CLAM protocol. It is re-run under our cost accounting, and we sweep the split and report the best split in hindsight so the baseline is not a straw man.

The strongest existing method is the Sadafi et al. pipeline: Monte Carlo dropout slide and attention uncertainty, the top slide queried, and its region mask requested. We run it on identical frozen features, the same MIL model and guiding loss, the same budgets and cost ratios, and the same number of retraining epochs. Its uncertainty weighting gets the same hyperparameter search budget as our policy's tuning. The only reported number for this method in our file is 0.72 ± 0.03 accuracy and 0.69 ± 0.03 F1 on CAMELYON17 after querying 6–8 slides (Sadafi et al.). That protocol is not ours, so we treat it as a sanity target for our re-implementation rather than a number to beat directly. For the outline effect, the reference numbers are 0.924 AUC and 0.350 Dice with random outlines on 10% of CAMELYON16 training slides (Mixed Supervision CLAM). We will count as beating a baseline a higher area under the cost–performance curve for the combined AUC-plus-Dice score, averaged over five seeds, with non-overlapping 95% bootstrap intervals, at two or more of three cost ratios.

We also include the bag-level heuristic AGIN from Carbonneau et al. adapted to attention models, used for slide queries only.

Three ablations isolate the proposal's ingredients. A bag-only policy removes the outline action. An attention-blind policy removes the attention-derived features. A myopic policy uses discount zero, so it greedily maximizes one-step gain. If the attention-blind policy matches the full one, our claim that attention behavior drives the outline decision is wrong.

Why not just…?

Why not just use a fixed schedule: slide labels first, then outlines once N slides are labeled? This is the real threat. If the benefit comes only from when to switch, a tuned threshold captures it without reinforcement learning. We include this schedule with N and the target-selection rule tuned by grid search on the target cohort's validation split, which gives it an advantage our frozen policy lacks. If it matches the learned policy, we will report that the granularity split, not the learned ordering, carries the gain. That is a useful negative result and would redirect the work toward better outline targeting alone.

Why not just use Monte Carlo dropout uncertainty, as Sadafi et al. do? Uncertainty flags slides where the model is unsure. The shortcut failure documented by mixed-supervision CLAM is a model that is confident and attends to stroma. Casanova et al. found uncertainty-driven selection wasteful under imbalance, and learned selection saved about 30% of labels over BALD. We test this directly. Among slides chosen for outlines, we measure the fraction whose pre-query attention Dice was below 0.1 under each method. If uncertainty already finds these slides, the two curves will coincide.

Why not just annotate a random 10% of slides with outlines? Random outlines already nearly double Dice (Mixed Supervision CLAM). If most outlines help roughly equally, selection adds little. The best-in-hindsight random split baseline settles this. At high cost ratios, we expect selection to matter because each outline must be justified.

Proposed Experiments

The primary test deploys the policy frozen on CAMELYON17, using the 50 slides with lesion annotations as the outline-capable pool and held-out centers for testing. Training uses only CAMELYON16 and MNIST-bags, so the test is also a cross-hospital transfer check, which addresses the known fragility of policies trained on one domain and applied to another. We sweep cost ratios $c_r/c_b \in \{10, 30, 100\}$ and budgets of 20–100 slide-label equivalents. We report cost–performance curves for AUC and Dice and the fraction of budget spent on outlines as a function of remaining budget.

On TCGA lung subtyping, which has no outlines, we run the bag-only policy. This tests whether the learned slide ordering transfers where the outline action is unavailable. Here the passive reference points are DTFD-MIL's AUC of 0.961 and CLAM's 0.956 with full labels, which the actively trained model should approach with far fewer labels.

Three checks will decide the question:

  • whether the full policy beats the tuned fixed schedule at two or more cost ratios on CAMELYON17;
  • whether the attention-blind ablation loses most of that gap;
  • whether the policy's outline choices concentrate on confidently mis-attended slides, measured against ground truth after the fact.

If all three hold, a learned policy is worth its complexity. If only the first fails, the contribution reduces to the diagnostic analysis of where outlines pay.