Many of the strongest immune biomarkers in cancer pathology are counts of immune cells inside a band of tissue called the invasive margin, the zone where tumor meets surrounding stroma. Lymphocytes crowding this band, or failing to cross it, predict survival. Nobody agrees precisely where the band is, though. Pathologists who draw it by hand disagree, and a margin shifted by a few hundred micrometers can move a patient between risk groups. We propose to stop treating the margin as a 2D contour and treat it as a 1D interval. Cut many short lines ("transects") perpendicular to the tumor edge. On each line, the margin is simply a start point and an end point. That is the same problem that single-shot temporal detectors solve when they find where an action starts and ends in a video. Because each line needs only two clicks per annotator, collecting five or more pathologists per slide becomes affordable. Because each line's margin is only two numbers, the spread of human opinion can be learned directly as a distribution rather than averaged away.
The core claim is testable. Margin-dependent immune scores computed from any single annotator's margin are unstable across annotators. A transect detector that predicts the distribution of plausible margins, and reports the immune score as an average over that distribution, gives a score that is both more stable and at least as prognostic. If a fixed-width band around an automatic tumor contour does just as well, the claim fails, and that outcome is informative too: it would mean margin placement does not matter for this biomarker.
What the three directions contribute, and what is actually open
The first direction supplies the method. Single-shot temporal detection predicts event boundaries and labels along an untrimmed 1D sequence in one forward pass. S3D tiles multi-scale anchor windows and regresses center and duration offsets. DOSED carries the same idea to EEG micro-events. Its authors note that such detectors are capped by how much human scorers disagree, yet they still train against a single consensus label. Gaussian Temporal Awareness Networks (GTAN) replaces rigid windows with learnable Gaussians, each with a center $\mu$ and a width $\sigma$, to handle fuzzy transitions. SegTAD argues instead for dense per-frame labeling, because assigning labels by overlap thresholds creates noisy supervision. We reuse the 1D detection machinery, and we take from GTAN the idea that a boundary should be a distribution. None of these works models how annotators disagree, and none has been applied to tissue.
The second direction supplies the target and the reason to care. Yang et al.'s colorectal study shows that lymphocyte density in stroma, together with tumor-stroma ratio, stratifies five-year survival. In their cohort the score gives HR 0.27 and an integrated AUC of 0.726, against 0.713 for clinical variables alone. The same study notes that patch-tiled CNNs break up the continuous gradients across the interface. Geller et al. and Pistilli et al. make the broader case that distance to structures matters more than bulk counts. These papers motivate the proposal, but all of them treat the compartment boundary as a fixed preprocessing output. So do the large spatial pipelines, the NSCLC spatial phenomics study and OSDR. No work in the file asks how much their conclusions move when the boundary moves.
The third direction supplies both the constraint and the unresolved tension. Variability Matters measured agreement among 120 pathologists on cell annotation and found median conformity of 0.70 and Fleiss' kappa of 0.43. Training on the most-conforming half of annotators beat training on everyone. DiFF argues the opposite: the right fusion of raters is the one that best serves the downstream diagnosis, so a rater who disagrees with the consensus may carry signal. Probabilistic models such as the Latent Doctor Model, variational inter-observer modeling and inter/intra-observer probabilistic segmentation learn a distribution over plausible 2D masks. All of this work concerns contiguous macroscopic lesions or individual cells, never a tissue zone whose width is itself uncertain. Multi-rater spatial annotations of tissue compartments also barely exist. We think the reason is cost: drawing a full margin contour on a whole slide takes a long time, so nobody collects many of them.
Core Claim
Disagreement about the invasive margin is mostly disagreement along one direction: how far the zone extends into tumor and into stroma, measured normal to the tumor edge. Reducing the margin to per-transect intervals does two things. It makes multi-rater annotation cheap enough to collect at scale. It also turns annotator disagreement into a low-dimensional quantity that a detector can predict and a biomarker can integrate over. The closest existing work either models 2D mask distributions (Latent Doctor), fuses raters for a classifier on macroscopic lesions (DiFF), or detects 1D events against a single consensus (DOSED). None of them predicts a distribution over margin zones in tissue, and none propagates that distribution into a survival biomarker.
Research Program
Building transects and collecting annotations. A standard 2D segmenter first produces a coarse tumor mask on each H&E whole slide image. Ordinary segmentation handles the bulk tumor well enough, so we are not trying to replace it. Along the mask contour we place transects every 100 µm, each 2 mm long and centered on the contour. Each transect becomes a 1D sequence of image-patch embeddings. Transects are ordered around the perimeter, which gives a second, along-contour axis.
Six pathologists annotate each sampled transect. Each marks either a start and end of the invasive margin, or "no margin present", for example where tumor abuts necrosis. On a subset of slides the same pathologists also draw full 2D margin contours. We time both protocols and treat the timing as a result to be measured, not assumed.
Detector. The detector is a 1D single-shot network that runs along each transect, with light attention across neighboring transects on the perimeter axis. It is trained end to end from the patch encoder up, following the evidence from ETAD and R-C3D that boundary localization benefits from adapting the backbone. For transect $j$, the detector outputs a small mixture of 2D Gaussians $p_\theta(s,e \mid x_j)$ over the margin's start $s$ and end $e$, plus a probability that no margin is present. Here $x_j$ is the transect's input sequence and $\theta$ the network weights. Training maximizes the likelihood of every rater's interval rather than their average:
$$\mathcal{L}(\theta) = -\sum_{j}\sum_{r=1}^{R_j} \log p_\theta\big(s_{jr}, e_{jr} \mid x_j\big),$$
where $R_j$ is the number of raters on transect $j$ and $(s_{jr}, e_{jr})$ is rater $r$'s interval. Averaging raters would collapse genuinely different opinions into a zone nobody drew. The likelihood keeps them apart, and it does so by construction rather than through an ad hoc confidence threshold.
Biomarker. A lymphocyte detector trained on PanNuke/Lizard-style nuclei data gives a lymphocyte density profile $\rho_j(d)$ along each transect, where $d$ is position along the transect. The margin immune score of a slide averages over both transects and plausible margins:
$$M = \frac{1}{J}\sum_{j=1}^{J} \mathbb{E}_{(s,e)\sim p_\theta(\cdot\mid x_j)}\left[\frac{1}{e-s}\int_s^e \rho_j(d)\,\mathrm{d}d\right],$$
where $J$ is the number of transects. The spread of the same quantity under $p_\theta$ gives each slide an uncertainty. Because the network models the start and end jointly, it can learn that some raters draw wide zones and others narrow ones, a pattern that a single 2D contour cannot express.
Tension analysis. The transect representation lets us test the disagreement between Variability Matters and DiFF directly. For each rater we compute the score $M^{(r)}$ from that rater's margins alone. We then ask whether low-conformity raters' margins carry survival information beyond the consensus, using a likelihood-ratio test in a Cox model.
Data. We will use TCGA cohorts overlapping those in ZTSurv, namely LUAD and BRCA, for which survival data and precomputed lymphocyte maps are available. Transect annotation will cover roughly 300 slides per cohort. Every slide gets six raters on at least 40 transects, and 60 slides also get full 2D contours.
Baselines
The naive reference is a fixed-width band, for instance 500 µm on each side of the automatic tumor contour, with lymphocyte density counted inside it. It needs no margin annotation at all and matches common practice. It is the most important competitor, because if the band matches our score on survival discrimination, margin localization is not worth modeling for this biomarker. We sweep the band width on training folds only, with the same cross-validation splits.
The consensus baseline trains a 2D U-Net on STAPLE or majority-vote fusions of the full 2D contours. It computes the score from the single predicted margin. It is trained on the same slides, with the same encoder initialization and the same hyperparameter search budget as our detector.
The strongest uncertainty-aware competitor is a 2D probabilistic multi-rater segmenter in the style of the Latent Doctor Model and probabilistic inter/intra-observer modeling. It is trained on the 2D multi-rater contours, sampled to produce many margins, and scored by averaging $M$ over samples. The file reports no numbers for these models on histopathology margins. It is the right competitor because it also represents disagreement, just not in 1D. We also run it on 2D masks rasterized from the transect annotations, which separates the cost of annotation from the choice of representation.
The rater-curation baselines apply the Variability Matters recipe, training only on the top 50% of raters by conformity. They also include a DiFF-style fusion whose rater weights are learned to maximize the Cox partial likelihood. That fusion is regularized as in DiFF and evaluated on held-out folds, to guard against the overfitting that paper warns of. DiFF reports AUC gains of up to 2.73% over standard fusion on REFUGE-2, but that result does not transfer to tissue margins.
As slide-level references we include weakly supervised multiple-instance learning, meaning a model that pools patch features into a survival prediction with no segmentation, and ZTSurv. ZTSurv reports an overall C-index of 0.661 across four TCGA cohorts. We will rerun it on our exact LUAD and BRCA splits rather than quote the pooled figure.
Ablations isolate the key ingredient. The first replaces the distributional head with a deterministic single-interval regression head in the style of DOSED, trained on the consensus interval. The second removes the along-contour attention. The third freezes the patch encoder. Beating the baselines means two things. First, a lower across-rater coefficient of variation of the slide score, measured when each held-out rater's margins serve in turn as the reference. Second, a C-index at least equal to the best baseline, with a paired bootstrap confidence interval that excludes a loss.
Why not just…?
Why not just use a fixed-width band around an automatic tumor contour? This is the real threat, and the proposal is built so it can win. The evidence from Yang et al. that stromal lymphocyte density matters does not show that the exact placement of the band matters. If the tuned fixed band matches our score on C-index and stability, we will report that margin placement is not prognostically important for lymphocyte density in these cohorts. The transect data set and the measured rater spread would remain useful products.
Why not just train a 2D probabilistic segmenter on multi-rater contours? In principle it can represent the same disagreement. Two differences matter. The first is cost: full multi-rater 2D margins are rarely collected, and the file notes that public data are overwhelmingly single-observer. The second is structure: a 2D mask distribution has no explicit notion of margin width. The rasterized-transect variant of this baseline separates these two effects. If it matches our detector, the contribution is the annotation protocol, not the 1D model, and we will say so.
Why not just take a majority vote or STAPLE consensus? Consensus yields one margin and therefore one score, so it cannot report which patients sit near a risk threshold. DiFF shows that uniform fusion loses diagnostic signal on macroscopic lesions. Our rater-level Cox analysis tests whether the same holds for tissue margins.
Why not skip segmentation and use multiple-instance learning on the whole slide? Such models may discriminate survival as well, but they cannot say whether lymphocytes sit in the margin or deep in stroma, which is the mechanistic question raised by Geller et al.. ZTSurv's reported advantage over patch-based baselines suggests structured tissue models are competitive. Our comparison on identical splits settles the point for these cohorts.
Proposed Experiments and Decisive Checks
The first experiment quantifies the problem. On the multi-rater slides we measure how often a patient's median-split risk group flips when the score comes from one rater's margins instead of another's. We also measure the per-slide coefficient of variation of $M^{(r)}$. If flips are rare, below about 5% of patients, the motivation weakens, and we shift emphasis to the width analysis.
The second experiment tests calibration against held-out raters. For each slide we hold out one rater and ask whether that rater's intervals fall inside the detector's predicted 80% region about 80% of the time. We report negative log-likelihood against the 2D probabilistic baseline, projected onto the same transects.
The third experiment is the prognostic test. We compare Cox C-index and hazard ratios across all baselines with five-fold cross-validation in each TCGA cohort. We also test whether the per-slide uncertainty identifies patients whose predicted risk is least reliable, by checking whether the model's discrimination is worse in the high-uncertainty tertile.
The fourth experiment measures annotation cost: minutes per slide for transect annotation versus full contours, and how agreement changes between the two protocols.
The decisive outcome combines all four. If the distributional transect score cuts across-rater variance substantially relative to the consensus U-Net, matches or beats the tuned fixed band and the 2D probabilistic segmenter on C-index, and loses its advantage when the distributional head is replaced by a deterministic one, then modeling disagreement as a distribution over 1D intervals is doing the work.
References
- S3D: Single Shot multi-Span Detector via Fully 3D Convolutional Networks
- DOSED: a deep learning approach to detect multiple sleep micro-events in EEG signal.
- Gaussian Temporal Awareness Networks for Action Localization
- SegTAD: Precise Temporal Action Detection via Semantic Segmentation
- Artificial intelligence for quantifying immune infiltrates interacting with stroma in colorectal cancer
- Spatial TIME landscape and its prognostic value in the lung and brain tumor: location matters
- Patritumab deruxtecan in HR+HER2− advanced breast cancer: a phase 2 trial
- AI-powered spatial cell phenomics enhances risk stratification in non-small cell lung cancer
- Temporal tissue dynamics from a spatial snapshot
- Variability Matters: Evaluating Inter-Rater Variability in Histopathology for Robust Cell Detection
- Calibrate the Inter-Observer Segmentation Uncertainty via Diagnosis-First Principle
- The Latent Doctor Model for Modeling Inter-Observer Variability
- Variational inference for quantifying inter-observer variability in segmentation of anatomical structures
- Probabilistic Modeling of Inter- and Intra-observer Variability in Medical Image Segmentation
- ETAD: Training Action Detection End to End on a Laptop
- R-C3D: Region Convolutional 3D Network for Temporal Activity Detection
- PointNu-Net: Keypoint-Assisted Convolutional Neural Network for Simultaneous Multi-Tissue Histology Nuclei Segmentation and Classification
- Cancer Survival Analysis via Zero-shot Tumor Microenvironment Segmentation on Low-resolution Whole Slide Pathology Images