When a machine-learning model for cancer slides asks a pathologist to outline suspicious regions, the system treats the pathologist's drawing as truth. Human attention is not a constant instrument, though. Cognitive science finds that people draw on one limited pool of attention and working memory: when one task gets harder, everything else running at the same time gets less. This proposal tests whether the omissions pathologists make while annotating gigapixel whole-slide images follow that pattern. The prediction is that, as load rises, missed lesions pile up on the small and low-salience ones. A second prediction is that the pathologist's own confidence rating does not register the decline, and that asking for the rating takes some of the same limited capacity.
If annotation errors are structured this way, a learning system can model them. A region the pathologist left unmarked during a high-load moment should not be treated as confidently negative, and the cost of asking for a confidence report should be counted against the annotation budget. The concrete deliverable is an annotation model that depends on the annotator's current load, fit from a controlled human study on CAMELYON slides with exhaustive reference contours. It is then plugged into attention-based active learning for lymph-node metastasis detection, with the same budget of annotator time as every comparison method.
Core Claim
Our claim has three parts. First, the probability that an annotator marks a true lesion depends on the interaction between their current attentional load and the lesion's size, not on either factor alone. Second, self-reported confidence tracks this decline much more weakly than accuracy does. Third, a training loss that reads unmarked regions through this fitted detection model gives better slide classifiers and lesion localization per minute of expert time than any of three alternatives: treating annotations as ground truth, weighting them by self-reported confidence, or using a generic annotator-noise model with no load term.
The load measure must not come from self-report. We use slowdown on a light concurrent tone-detection probe. This follows the cross-modal suppression results showing that harder visuospatial tracking suppresses responses to unattended sounds (Roth et al.; Callan et al.). In these designs, performance on the secondary channel acts as a meter of how much capacity the main task is consuming.
The detection model is a logistic regression. Its symbols are as follows:
- $d_{i}$ is whether lesion $i$ is outlined.
- $s_i$ is the lesion's area.
- $L_t$ is the probe-derived load at the moment the slide was viewed.
- $\alpha_a$ is annotator $a$'s baseline skill.
- $\beta$, $\lambda$ and $\mu$ are fitted coefficients.
$$\operatorname{logit} P(d_i = 1) = \alpha_a + \beta \log s_i - L_t\,(\lambda - \mu \log s_i)$$
The resource-competition account predicts $\mu > 0$, meaning load harms small lesions disproportionately. This mirrors the finding that low-priority working-memory items lose neural gain and precision first (Sprague et al.). A generic noise model corresponds to $\lambda = \mu = 0$.
Relation to Prior Work
The pathology side supplies the method and also motivates the gap.
- Sadafi et al. query experts for regions of interest using attention uncertainty. They add an attention-guiding loss that penalizes model attention outside expert regions. Their experts were simulated from pre-annotated CAMELYON17 contours, so the annotations were effectively perfect. We reuse their loss and query scheme, but replace perfect simulated experts with measured, load-dependent ones.
- HAG-MIL zooms from low-magnification attention into top-$k$ high-resolution patches "like a pathologist." That is machine attention inspired by human scanning, not a measurement of human attention. We use it only as a slide-classifier reference.
- AttriMIL shows that high attention weight does not say whether a patch argues for or against cancer. This constrains our design: guiding attention toward annotated regions is useful only for the positive evidence those regions represent. For that reason we soften penalties on unmarked areas rather than forcing attention onto marked ones.
- GleasonXAI is the closest work on human fallibility. It trains on the full distribution of 54 pathologists' Gleason annotations rather than a majority vote. It treats disagreement as a fixed property of each image. We ask whether a large share of disagreement is instead a property of the annotator's state at the moment of annotation, which no fixed per-image distribution can capture.
The cognitive side motivates both the structure of the errors and the possibility that self-report is costly.
- Active mental construction displaces stored items under high memory load (Wang et al.).
- Effortful retrieval can exhaust the budget needed for re-encoding (Sun and Liu; Shi and Liu).
Together these suggest that composing a confidence rating or rationale while a slide is still being searched is not free. The self-report literature motivates distrusting the rating itself. Workplace self-assessments drift toward perceived competence (new-nurse survey), and retrospective reports are reconstructed rather than replayed (RECOVER-Adult). The micro-randomized trial in medical interns by NeCamp et al. found that prompts delivered when people were doing well lowered their reported mood (moderation estimate −0.052, P=.001). That is evidence that the act of prompting changes the state being measured, though in a different domain. We treat all of this as a hypothesis to test in pathology, not as an established fact there.
Research Program
Study 1: controlled annotation study. Pathology residents and attending pathologists outline metastases on CAMELYON16 and CAMELYON17 slides. Reference truth comes from the exhaustive hand-drawn metastasis contours released with the CAMELYON dataset (209 contoured WSIs). We plan roughly 25 readers with about 40 slides each, balanced across macrometastases, micrometastases and isolated tumor cells.
Load is manipulated within readers across blocks using time pressure and position in a long reading queue. It is measured continuously with the tone-probe slowdown. A second within-reader factor crosses blocks with and without a post-slide confidence rating and one-line rationale. The design is preregistered and tests three things:
- The load × size interaction ($\mu > 0$).
- A flatter slope of confidence than of lesion-level sensitivity against load, scored by calibration error.
- Lower small-lesion sensitivity in confidence-reporting blocks at matched load.
Study 2: load-aware training. The fitted detection model turns each annotation into a soft target. Marked regions keep their label. Each unmarked region receives a posterior probability of containing tumor, computed from its size, the annotator, and $L_t$ via Bayes' rule with a slide-level prior. The attention-guiding penalty of Sadafi et al. is then scaled down by that posterior, so the model is not punished for attending to tissue a loaded annotator may simply have missed.
Study 3: active learning in simulation. We simulate active learning on CAMELYON17, replacing perfect simulated experts with annotators sampled from the fitted model. Budget is counted in annotator minutes, including time spent on confidence reports. The goal is to see whether the effect holds at scale and whether a query policy that also chooses when to ask (deferring small-lesion candidates away from high-load moments) adds value beyond the training loss.
Baselines
Naive reference: annotations as ground truth. This is Sadafi et al.'s S-MIL-AGL trained on the real Study 1 annotations, taken as given. It is the strongest existing method for this exact question. With perfect simulated experts it reports CAMELYON17 accuracy 0.72 ± 0.03 and F1 0.69 ± 0.03, against 0.33 ± 0.27 and 0.31 ± 0.30 for standard MIL (Sadafi et al.). We expect it to fall below those numbers once annotations come from real loaded readers. No reported result was found on Lacuna for any pathology active-learning method trained on human annotations collected under measured load.
All methods share the same backbone features, the same slide splits, the same annotation pool and minutes budget, and the same hyperparameter search budget. The perfect-annotation version of S-MIL-AGL is also reported as a ceiling.
Confidence weighting. Each annotation is weighted by the reader's self-reported confidence. This is the obvious way to bring self-report into training.
Static annotator-noise model. We use per-annotator detection rates with a size term only ($\lambda = \mu = 0$). We also run a soft-label variant in the spirit of GleasonXAI that pools readers' outlines into a per-slide probability map. GleasonXAI reports Dice 0.713 against 0.691 for hard labels on its prostate cohort (GleasonXAI). This is the ablation that isolates the key ingredient: if it matches the full model, the cognitive load term adds nothing.
Scheduling-only. Sessions are capped and breaks are inserted, with naive training. This tests whether simply avoiding load is enough.
Slide-classifier reference. HAG-MIL is reported on CAMELYON16 (AUC 0.946, F1 0.874 in Xiong et al.) to show that the shared backbone is competitive.
What counts as beating the baselines. At equal annotator minutes, the load-aware model must exceed both the naive and static-noise baselines on CAMELYON17 slide-level F1 and on lesion-level localization sensitivity for micrometastases. The margins must exceed the seed-to-seed standard deviation across five seeds and hold on held-out readers.
Why not just…?
Why not just weight labels by the pathologist's confidence rating? Confidence is exactly the quantity the self-report literature tells us to doubt: professional self-assessment inflates (new-nurse survey), and prompting can alter the state it measures (NeCamp et al.). Neither result comes from pathology, so we do not assume the answer. Study 1 measures directly whether confidence tracks small-lesion sensitivity under load and whether collecting it lowers sensitivity. The confidence-weighting baseline then shows the downstream consequence. If confidence turns out to be well calibrated and free to collect, this baseline will win, and we will report that.
Why not just use a generic annotator-noise or soft-label model? This is the real threat. Per-annotator noise models and GleasonXAI-style soft labels already improve on hard labels without any cognitive theory. The static-noise ablation settles it. If $\mu$ is not reliably positive, or adding $L_t$ does not improve held-out prediction of omissions and downstream F1, the cognitive component reduces to ordinary noise modeling. The publishable finding then becomes that annotation omissions are not load-structured.
Why not just give pathologists breaks and keep sessions short? Scheduling may remove most high-load moments cheaply, and we test it as a baseline. It cannot address load that comes from the slide itself: a dense, artifact-heavy slide is demanding at minute one. It also spends expert time. If scheduling matches the load-aware loss at equal minutes, the practical recommendation is scheduling, and the modeling contribution reduces to explaining why it works.
Decisive Checks
The project stands or falls on Study 1's interaction term. Lesion-level omissions must show $\mu > 0$ with a confidence interval excluding zero on held-out readers. The probe-based load measure must predict omissions beyond queue position alone, which rules out simple time-on-task drift.
Downstream, the full model must beat the static-noise ablation on micrometastasis localization at equal minutes. Small lesions are where the theory says the gain should concentrate, so a gain only on macrometastases would count against the account.
A final falsification check is that the confidence-reporting manipulation lowers small-lesion sensitivity at matched load. If it does not, the "self-report is costly" part of the claim is dropped, and the proposal rests on load-structured omission alone.
References
- The influence of visuospatial attention on unattended auditory 40 Hz responses
- The role of brain-localized gamma and alpha oscillations in inattentional deafness: implications for understanding human attention
- Neural mechanisms of resource allocation in working memory
- Active Learning Enhances Classification of Histopathology Whole Slide Images with Attention-Based Multiple Instance Learning
- Diagnose Like a Pathologist: Transformer-Enabled Hierarchical Attention-Guided Multiple Instance Learning for Whole Slide Image Classification
- Rethinking Attention-Based Multiple Instance Learning for Whole-Slide Pathological Image Classification: An Instance Attribute Viewpoint
- Pathologist-like explainable AI for interpretable Gleason grading in prostate cancer
- Lie construction affects information storage under high memory load condition
- Retrieval practice is costly and is beneficial only when working memory capacity is abundant
- A working memory dependent dual process model of the testing effect
- The effects of work readiness, organizational justice and professional identity on the work performance of new nurses: a cross-sectional survey
- Incidence and Prevalence of Post-COVID-19 Myalgic Encephalomyelitis: A Report from the Observational RECOVER-Adult Study
- Assessing Real-Time Moderation for Developing Adaptive Mobile Health Interventions for Medical Interns: Micro-Randomized Trial
- 1399 H&E-stained sentinel lymph node sections of breast cancer patients: the CAMELYON dataset