Tracking Mitotic Figures Through Focus

When pathologists grade a meningioma, a brain-surface tumor, one of the deciding numbers is the mitotic count: how many dividing cells appear in a fixed area of tissue. Experts often disagree on this count, and a large share of that disagreement is physical rather than cognitive. A standard slide scan captures one focal plane. A dividing cell whose chromosomes sit slightly above or below that plane can look like a dying cell or a smudge of debris. At the microscope, pathologists settle these cases by turning the focus knob. Z-Stack Scanning can Improve AI Detection of Mitosis showed that scanning five focal planes (a "z-stack") raises detector sensitivity from 0.601 to 0.704 on average with precision unchanged. Their pipeline, however, treats each plane separately. It merges candidates by simple box overlap and then votes over 20 classifier outputs per candidate.

This proposal treats the focal axis the way video trackers treat time. Each candidate figure becomes a short "focus track": the same object seen at several depths. The system then decides jointly which per-plane detections belong to one physical object, which belong to different objects stacked at different depths, and whether the object is a mitosis. The claim is that this joint decision matters most for counting. It prevents one figure from being counted twice across planes, and it keeps two overlapping figures from being merged into one. Counting errors are what push a tumor across a grade threshold. We also claim that the cases the joint model finds ambiguous are the ones human observers disagree on, and that a large part of that disagreement goes away once observers can see the focal stack.

Core Claim

Our hypothesis is that mitotic-count discordance in single-plane scans comes from two kinds of focus-dependent error. One is appearance ambiguity: mitosis versus a mimic. The other is identity ambiguity: one object versus two. Z-stack information fixes the first even with naive fusion. Fixing the second requires grouping detection hypotheses across planes and space jointly, instead of suppressing overlaps within each plane first. The hypothesis predicts two things. First, a joint cross-plane partition should reduce count error and grade-threshold crossings beyond what simple fusion achieves, even at matched per-object F1. Second, its gains should concentrate in the cases where raters disagree when viewing one plane but agree when given the stack.

Research Program

The per-plane component is deliberately simple. We will use a segmentation-style detector that sees all five registered planes as input, so it gets depth context cheaply. It will output dense per-plane candidate hypotheses with no non-maximum suppression, the standard step that deletes overlapping boxes.

We borrow the association step from Subgraph decomposition for multi-target tracking. That work replaced per-frame suppression plus frame-to-frame linking with a single graph partition over detection hypotheses spanning many frames. The partition used consistency constraints: if A and B are the same object, and B and C are the same object, then A and C are too. On TUD-Crossing it raised tracking accuracy (MOTA) from 71.8% to 80.9% and cut identity switches from 18 to 1. In our setting, the nodes are candidate figures in each of the five planes. The edges connect hypotheses within a plane and in neighboring planes. A small learned network scores each edge using appearance embeddings, centroid offset, and the change in sharpness between planes.

Two design choices follow from the constraints in the file. That paper reported that exact integer programming took 22 hours versus 12 seconds for a local-search heuristic, so we will use the heuristic solver on per-field graphs, which are small. CellTrack R-CNN jointly segments and tracks dividing cells in video, but it encodes spatial context with a fixed set of four nearest neighbors. Tissue density varies too much for that, so we use a radius-based neighborhood instead.

Each resulting group is a focus track, and a small sequence model classifies it using the ordered stack of crops. The classifier is trained as regression on observer agreement rather than on binary labels. This follows Considerations for data acquisition and modeling strategies, which found that regressing crowd agreement gave a better trade-off between true and false positives than box detection. That work used only single-plane tiles, and we extend it to the focal axis. Edge scores, detector, and track classifier are trained together through a relaxed version of the partition loss. We do not claim that full end-to-end training is necessary, and that question is tested as an ablation.

The observer-variability component is a measurement study. We will take a stratified subset of candidates from the z-stack data and have several pathologists label them twice, once with a single plane and once with scrollable access to all five planes. A cheaper panel will follow the gaze-aggregation protocol of Supporting Mitosis Detection AI Training with Inter-Observer Eye-Gaze Consistencies. That paper aggregated fixations from 14 non-expert observers and reached 0.83 average precision, close to 0.85 for expert labels, at 2.63 seconds per image. We use such panels as the agreement targets and as a disagreement map. This study also tests whether the "focus gap" really explains discordance, which the z-stack paper argued but did not measure against raters.

Baselines

The naive reference is a single-plane detector trained and tested on the nominal focal plane only. On this data the z-stack paper reports an average sensitivity of 0.601. This bounds how much depth information contributes at all.

The strongest existing method is the decoupled z-stack pipeline from Z-Stack Scanning can Improve AI Detection of Mitosis. It segments each plane independently, merges candidates by overlap, runs a four-network ensemble on every plane, and aggregates the 20 outputs with a random forest. It is the strongest competitor because it already uses all five planes and a large ensemble. It reports average sensitivity 0.704 and precision 0.757, with a best sensitivity of 0.773 for DeepLabV3+ on the Aperio GT 450 scanner. We will re-run it on identical splits across the three scanners, with the same total GPU-hour tuning budget as our method. Beating it means one of two things: higher F1 at no more inference cost, or equal F1 with significantly lower count error per field and fewer grade-threshold crossings. The second outcome is the one our claim is about.

The focus-fusion baseline collapses the stack with an all-in-focus composite and runs a 2D detector. For this we found no reported result on Lacuna for all-in-focus mitosis detection. We will also run an early-fusion baseline that uses our own per-plane detector with standard suppression and returns one box per location, without the partition.

Three ablations isolate the key ingredient. The first replaces the partition with suppression followed by overlap merging. The second replaces agreement regression with binary labels. The third freezes the detector and trains only the edge scorer and classifier, which tests whether joint training adds anything beyond the partition itself.

Why not just…?

Why not just fuse the planes into one all-in-focus image and use a normal detector? The z-stack paper itself names this as the obvious way to recover its 3.81× storage cost. It is cheap and will likely capture much of the appearance gain. What it cannot represent is two figures at different depths that overlap in the 2D projection, because the composite superimposes them. Our count-error metric, stratified by whether a reference object has an xy-overlapping neighbor in another plane, measures exactly this. If fusion matches us on count error in those strata, the partition is unnecessary and we will say so.

Why not just stack the planes as input channels and keep standard suppression? This is the most serious threat, because the planes are already pixel-registered and a network can learn depth cues directly. We adopt it as our detector input, so the comparison isolates the partition step alone. If early fusion with suppression ties our method on both F1 and count error, the joint-grouping claim fails. The remaining contribution would then be the rater study and the agreement-regression targets.

Why not just treat disagreement as label noise and train on more consensus labels? Consensus voting cannot tell a rater who lacked focus information apart from one who applied the criteria differently. Considerations for data acquisition and modeling strategies already found that modeling the agreement continuum beats forced consensus boxes. The paired single-plane versus full-stack rating study settles the question directly. If agreement barely improves with stack access, discordance is mostly about criteria, and our method's relevance to observer variability shrinks to a detection improvement.

Proposed Experiments

The main benchmark is the five-plane KUMC meningioma z-stack set used in the z-stack paper: 22 slides, 6,350 mitoses, three scanners. We will use leave-one-scanner-out splits. This tests the domain-shift concern raised by the MIDOG challenge, where scanner optics degraded all detectors; its winning single-plane F1 of 0.748 serves only as context, since the data differ. Metrics are per-object F1, sensitivity, and precision; absolute count error per ten high-power fields; the fraction of fields whose count crosses the WHO grading threshold relative to reference; and inference cost per field. We will also check single-plane transfer by collapsing our model to one plane and evaluating it on stMIDOG++. There, A Subphase-Labeled Mitotic Dataset reports F1 0.794, and its telophase and atypical-figure labels let us see whether the partition avoids fragmenting dividing figures.

The decisive checks are three. The first is whether our method reduces count error relative to early fusion plus suppression, specifically in the xy-overlap strata. The second is whether the paired rater study shows inter-observer agreement rising with stack access. The third is whether the model's predicted agreement correlates with single-plane rater disagreement more strongly than the single-plane baseline's scores do. If the first check fails, the joint formulation is not needed. If the second fails, the observer-variability motivation is wrong.