Reading Slides on a Budget: Molecular Data as a Training-Only Critic for Tile Selection

Predicting tumor genetics from a routine stained tissue slide is now common practice in research. Microsatellite instability (a DNA-repair defect) in colorectal cancer and homologous recombination deficiency (another DNA-repair defect) in breast cancer are two examples. The models that do this assume something rarely true in a clinic: that every one of the tens of thousands of high-magnification tiles in a gigapixel whole-slide image (WSI) is scanned, stored, and processed. Molecular measurements make this worse because they are expensive and destructive, so they are available only for a training cohort and never for the new patient. This proposal treats that asymmetry as the main design fact. A cheap image is all we have at deployment, and rich molecular and full-resolution information is available only during training. The question is how to use that training-only information to teach a model where to look when it can afford to read only a small fraction of a slide.

The method borrows a recipe from multi-agent reinforcement learning called centralized training with decentralized execution (CTDE). During training, a "critic" network sees everything. Here that means the full-resolution slide plus the patient's measured molecular profile. The critic scores the choices of several simple "reader" agents. Each reader sees only a low-magnification view of its own region of the slide and a fixed budget of high-magnification tiles it may request. At test time the critic is thrown away. The readers run independently and in parallel on hardware-limited scanners, and none of them has seen any molecular data. The core claim is that a critic with access to molecular data can give each reader counterfactual credit, meaning how much the molecular prediction would have changed had it requested a different tile. This credit yields tile-selection policies that reach full-slide accuracy at a small fraction of the scanning cost. It does so better than selection derived from attention maps, because attention reflects correlation with the label and not marginal contribution.

Core Claim and What Is New

Two literatures meet here without having touched before. Computational pathology has built strong slide-level molecular predictors. Three of them illustrate the range: attention-based learning over many tiles with adversarial removal of hospital artifacts (CDA-CLAM), transformers with sparse local attention over up to 1,600 sampled tiles (SuRe-Transformer), and clustering a slide down to 49 representative tiles before regressing bulk RNA expression (tRNAsformer). These systems motivate the proposal. All of them pick tiles either exhaustively or with label-agnostic heuristics such as k-means clusters and cluster-size weighting, and none of them treats which tiles to acquire as a decision with a cost.

Multi-agent reinforcement learning supplies the method. MADDPG introduced critics that condition on global state while actors see only local observations. COMA introduced the counterfactual baseline. That baseline subtracts the expected value over an agent's alternative actions, with the other agents' actions held fixed:

$$A^a(s,\mathbf{u}) = Q(s,\mathbf{u}) - \sum_{u'} \pi^a(u' \mid \tau^a)\, Q\big(s,(\mathbf{u}^{-a},u')\big),$$

Here $s$ is the privileged state (full-resolution tile features plus the molecular profile), $\mathbf{u}$ is the joint set of tile requests, $\tau^a$ is reader $a$'s local low-magnification observation, and $\pi^a$ is its request policy.

What is new is the content of $s$. Prior CTDE work uses simulator state. Here the privileged state is the molecular measurement itself, so the side of the morphology–molecule gap that is missing at deployment becomes the training signal for acquisition.

We use the COMA-style policy gradient rather than QMIX on purpose. QMIX requires the joint value to increase monotonically in each agent's utility. Tile value is not monotonic in that way: a second lymphocyte-rich tile is worth little if a neighboring reader already captured one, because the two are redundant. The monotonicity limitation documented for QMIX is therefore a real constraint on this design and not an aside.

The resource side is handled in the spirit of orthogonal resource allocation. In orthogonal multiple access, wireless resources are split into disjoint blocks so users never interfere. Here each reader gets a disjoint, fixed tile budget, which is what makes execution truly decentralized and parallel. The "non-orthogonal" alternative is a shared budget pool that readers draw from. It is tested as an ablation, not assumed. The wireless literature (Ding et al.; the bandwidth saturation reported over OFDMA in CKAN-HAPPO) shows that rigid partitions waste capacity when demand is uneven. Tumors are uneven, so we expect the same trade-off and will measure it.

Research Program

The system has three components. A frozen tile encoder uses a self-supervised pathology backbone of the kind used by SuRe-Transformer and by the CTransPath-based MSI detector of Gustav et al.. A small shared reader policy maps a low-magnification region thumbnail and the reader's past requests to a distribution over candidate high-magnification tiles in its region. A slide-level aggregator, available at test time, turns the requested tiles into a molecular prediction.

The training-only critic receives the full set of high-magnification tile features, the patient's molecular profile, and the joint requests. It predicts the aggregator's log-likelihood of the true label. When bulk RNA is available, as in TCGA, it also predicts the aggregator's error on a bulk expression signature. We include the expression signature because Arslan et al. report that RNA and protein are far more visible in morphology than DNA mutations. A richer molecular target should therefore give the critic a denser and more learnable signal than a binary mutation label alone.

Since the tile features are fixed, the counterfactual sums are cheap. Evaluating the critic on alternative tiles requires no rescanning.

The analysis has two parts. First, which tiles do policies trained this way request, and how does that compare with attention-ranked tiles? Second, do the selected tiles carry hospital identity? CDA-CLAM showed that baseline attention models identify the source hospital almost perfectly and lose external accuracy as a result. We will add the same adversarial domain discriminator to the aggregator and test whether a selection policy trained with a molecular critic avoids artifact-heavy tiles on its own.

Baselines

The naive reference is uniform random sampling of the same number of high-magnification tiles per region, fed to the same aggregator. Every method's gain is measured against this at matched budgets.

The ceiling is a full-slide model that reads every tile. For colorectal MSI/POLE we use CDA-CLAM retrained on our cohorts. Its reported independent-test ternary macro-average AUC is 0.9323, with external class-wise AUCs of 0.9674 for POLE and MSI-H (CDA-CLAM). For breast HRD the ceiling is SuRe-Transformer, which reports an AUROC of 0.887 ± 0.034 internally and 0.750 on the external ChosenMed-BRCA cohort (SuRe-Transformer). For MSI on TCGA, Gustav et al. report an external AUROC of 0.87. These published numbers are anchors only. All comparisons will be re-run on our splits, with the same frozen encoder and the same hyperparameter-search budget for every method.

The strongest competitor for this specific question is attention-guided selection. We train an attention-based multiple-instance model on the full slide. We then train a lightweight predictor that maps the low-magnification thumbnail to the full model's attention scores. At test time it requests the top-ranked tiles within the same per-region budget. This is the obvious engineering answer, and it uses the same privileged full-resolution data at training time.

Label-agnostic selection, meaning k-means representatives as in tRNAsformer and cluster-size-weighted sampling as in SuRe-Transformer, forms a second family. It is likewise restricted to the same budget.

Beating the strongest competitor means one of two things. Either the method reaches at least 95% of the full-slide AUROC at a tile budget at most half the size attention-guided selection needs for the same AUROC, or it achieves an external-cohort AUROC drop that is smaller with non-overlapping bootstrap intervals.

Three ablations isolate the key ingredients. The first removes the molecular profile from the critic, leaving a full-resolution-only critic, to test whether the privileged molecular state matters. The second replaces the counterfactual advantage with the shared team reward, to test whether per-reader credit matters. The third swaps disjoint per-reader budgets for a shared pool, to measure the cost of orthogonal partitioning.

Why not just…?

Why not just distill a full-slide teacher, or a teacher trained with molecular data, into a cheap student? This is the most serious threat. Training with privileged information through distillation is simpler than reinforcement learning. Distillation, however, shapes what the student predicts from the tiles it is given. It does not decide which tiles to acquire. The attention-guided baseline is exactly "distill where the teacher looks." If it matches our method at every budget, the counterfactual critic adds nothing, and we will say so. The first ablation separates the two explanations. If a critic without molecular inputs does as well as the full critic, the gain comes from full-resolution privilege, which distillation can also exploit, and not from molecular privilege.

Why not just use leave-one-tile-out occlusion to score tile importance and pick the top tiles? Occlusion measures each tile's marginal effect against the full slide. Under a small budget, what matters is a tile's value given the other few tiles chosen. Redundancy between neighboring tumor-infiltrated tiles is the typical failure, and that is precisely the non-monotonic interaction the COMA baseline conditions on through $\mathbf{u}^{-a}$. We will run occlusion-ranked selection as an additional baseline. If it ties at all budgets, our joint credit assignment is unnecessary.

Why not just use a single sequential agent instead of several decentralized readers? A single agent with global memory is strictly more expressive. It forfeits parallel scanning and has to reason over sequences hundreds of steps long. The skeptic may be right on accuracy. We will train a single-agent version with the same total budget and report both accuracy and wall-clock acquisition time under a simulated scanner. The multi-reader design is justified only if it loses little accuracy while gaining parallelism.

Proposed Experiments

The primary benchmark is colorectal MSI-H/POLE/MSS classification. We train on TCGA colorectal slides and the DACHS cohort used by Gustav et al., and validate externally on held-out centers. The multi-center POLE cohort of CDA-CLAM will be used where access permits.

The secondary benchmark is breast HRD on the SuRe-Transformer cohorts, with ChosenMed-BRCA as the external test. On TCGA kidney, we use tRNAsformer's bulk RNA targets to test whether a richer privileged critic helps more than a binary label. We can also test whether the selection policy transfers to the OSU external kidney cohort, where tRNAsformer reports over 82% subtyping accuracy.

The main result is a budget curve: AUROC against the fraction of high-magnification tiles read, at 1%, 2%, 5%, and 10%, for every baseline. We will also report the external-minus-internal AUROC gap. As a diagnostic of artifact exploitation, we will report the accuracy of a hospital classifier trained on the selected tiles alone.

The decisive check is a two-by-two design: molecular versus non-molecular critic, crossed with counterfactual versus shared reward, evaluated at a 2% budget on the external cohort. The proposal stands only if the molecular counterfactual cell beats the other three cells and also beats attention-guided selection. If the gains disappear on biopsies, where Gustav et al. already observed degradation, that boundary will be reported as a limit of the approach. It will not be averaged away.