Plants are most fragile in short developmental windows. A heat or drought pulse that a seedling shrugs off can be devastating a day later, once new organs have begun to form. Computer-vision phenotyping now estimates these windows by segmenting plant organs in time-lapse images and recording when each organ first appears. Those segmentation models learn from human-drawn outlines. Human annotators disagree with each other, and the disagreement is probably not uniform. It is likely to be largest when the newest organs are tiny and ambiguous, which is also when the plant is moving into a new stage.
This note proposes to test whether that disagreement is a timing bias rather than random noise. The hypothesis is that consensus labels systematically detect organ emergence late, and that this lag is large enough to put a stress event in the wrong developmental window. The fix is a model of each annotator whose bias is allowed to depend on developmental stage. To calibrate it, the method uses a reference that needs no extra annotation: organs only grow, so a leaf that is unmistakable at time $t+k$ must already have existed at time $t$. Later frames therefore reveal what earlier annotators missed.
Core Claim
Inter-observer variability in plant segmentation changes with developmental stage. At transitions it is dominated by omission of small emerging organs, not by symmetric boundary jitter. Averaging annotators therefore biases the estimated time of stage onset toward the late side.
The first piece of evidence is the observer study Citizen crowds and experts: observer variability in image-based plant phenotyping. It collected leaf counts from experts, non-experts and more than 5,000 Zooniverse citizen annotations on Arabidopsis. It found that higher image resolution helped experts catch emerging leaves but did not help non-experts, who missed them regardless. That is a direction-specific error, which averaging cannot cancel.
The second piece is the size of the windows at stake. Ma et al. show floral drought transcriptomes with almost no overlap between tissues differing by about 24 hours in age. A one-day lag in detecting a stage transition is therefore biologically meaningful, not cosmetic.
The proposal claims three things. A rater model whose bias depends on stage, calibrated against temporal hindsight, will reduce the lag in onset estimates. It will do so in the online setting where hindsight is unavailable. And it will improve the assignment of stress events to developmental stages, beyond what consensus training or post-hoc temporal smoothing achieve.
Relation to Prior Work
The same observer study both motivates and constrains this proposal. It motivates it through the resolution paradox. It constrains it because the averaged counts of three citizens were statistically indistinguishable from experts when tracking growth differences between wild-type col-0 and starch-deficient pgm plants. Simple consensus is therefore a serious competitor.
The two claims are compatible. That study measured whole-trajectory differences between two genotypes, where a constant lag cancels out. This proposal targets onset timing, where a lag is the whole error. The earlier study also used counts and dots rather than pixel outlines, and imposed no stress.
The method borrows from probabilistic rater modeling. Pionono (Probabilistic Modeling of Inter- and Intra-observer Variability in Medical Image Segmentation) gives each annotator a Gaussian in a latent space. Its mean captures that annotator's systematic bias and its spread captures their self-inconsistency, and a U-Net conditioned on these samples outputs annotator-specific masks. Related variational and Bayesian formulations are The Latent Doctor Model for Modeling Inter-Observer Variability and Inter-Rater Uncertainty Quantification in Medical Image Segmentation via Rater-Specific Bayesian Neural Networks. All of these treat each annotator's behavior as fixed across images and were validated on static medical images. This proposal makes rater behavior a function of the organ's size and age, and validates it against a time-derived reference that static images cannot provide.
Calibrate the Inter-Observer Segmentation Uncertainty via Diagnosis-First Principle (DiFF) chooses the ground truth that maximizes downstream diagnostic AUC. This proposal shares that idea of judging labels by their downstream use. It differs in that its downstream quantity, onset time, can be checked directly against hindsight. It therefore does not need a learned utility network, which DiFF itself reports is prone to high-frequency artifacts.
The segmentation backbones come from plant work that assumes a single correct outline. ChronoRoot 2.0 (ChronoRoot 2.0, code at ChronoRoot2) adapts nnU-Net, a self-configuring medical segmentation network, to six-organ time-lapse imaging every 15 minutes. It smooths flicker with a trailing average. GMT: Guided Mask Transformer for Leaf Instance Segmentation adds growth-from-the-center priors precisely because Mask2Former misses small central leaves. Both train on single-consensus masks. This proposal asks what those masks cost at transitions.
Research Program
The first component is measurement. Three expert and three non-expert annotators will outline leaves and organs on time-lapse sequences, oversampling frames near transitions. The sequences come from two sources: ChronoRoot-style Arabidopsis and tomato seedling imaging, and rosette sequences from the Phenotiki/Zooniverse Arabidopsis material. Following the observer study's protocol, repeated frames will be shown rotated and flipped so that self-inconsistency can be measured alongside disagreement between annotators.
The hindsight reference is built as follows. Each organ is tracked backward from the first frame where all annotators agree it exists. The earliest frame where its tracked footprint is visible above a fixed pixel threshold is taken as its reference onset.
The main analysis models disagreement and omission as functions of organ area and of time since the reference onset. The test is whether omission dominates near onset and decays as the organ grows.
The second component is the model. A Pionono-style network is kept, but the rater-specific probability of missing an organ is allowed to depend on the organ's apparent size and the developmental context:
$$p_r(\text{miss} \mid a, s) = \sigma\big(\alpha_r + \beta_r^{\top} h(s) - \gamma_r \log a\big),$$
Here $a$ is the organ's pixel area, $s$ is the stage (number of established organs) inferred from the image, $h(s)$ is a small learned embedding of $s$, and $\sigma$ is the logistic function. The parameters $\alpha_r$, $\beta_r$ and $\gamma_r$ are per-rater offset, stage sensitivity and size sensitivity. Boundary jitter on detected organs stays in Pionono's Gaussian latent.
Training fits these parameters to multi-rater labels while treating the hindsight reference as partial supervision on existence. At test time the network outputs the de-biased probability that a small organ is present, using only frames up to the current time. This is the online condition a stress-alerting system would face.
The third component is the downstream consequence. Short heat pulses will be applied to seedlings in the ChronoRoot imaging setup at staggered times around the emergence of the first true leaves. Each plant's stage at stress will be assigned from each model's onset estimates. The question is how well that assigned stage explains later growth trajectories, summarized with ChronoRoot's functional principal components.
Prior work says vulnerability differs sharply across stages (Chan et al.; Jin et al.; Shi et al.). If so, correct stage assignment should explain more of the response variance. Stage-misassignment rate against the hindsight reference is the direct measure.
The scope is deliberately limited to seedling and rosette stages, where emergence is visible. The file notes that late-stage canopy occlusion defeats current tracking. Reproductive windows in field crops are therefore the motivation, not the test bed.
Baselines
The naive reference point is a segmentation backbone trained on a single annotator's masks, or on majority-vote masks, with onset read off as the first frame of detection. Two backbones will be used: nnU-Net as configured in ChronoRoot 2.0, and GMT for rosettes. On CVPPP leaf segmentation, GMT reports symmetric best Dice of 90.1 and absolute count difference of 0.48, against Mask2Former's 89.5 and 0.67. This makes it the strongest static leaf segmenter in the file. ChronoRoot reports multi-class Dice above 0.80. No reported result was found on Lacuna for onset-timing error or stage-assignment accuracy of any of these models, so this proposal must establish those numbers itself.
The strongest consensus competitor is the observer study's three-citizen average, alongside STAPLE fusion. The strongest rater-modeling competitor is static Pionono, which reports Cohen's kappa of 0.758 on Gleason 2019 and 0.711 on the breast-cancer set. Its code and architecture are reused unchanged except for the stage-dependent omission term.
All methods share the same images, splits and backbone. Each gets the same hyperparameter search budget, here 20 configurations selected on a validation split. Compute is matched.
The ablation that isolates the key ingredient sets $\beta_r = \gamma_r = 0$, which recovers static Pionono. A second ablation drops the hindsight supervision.
Beating the baselines means two things. Online onset error must drop by a margin larger than the inter-annotator spread on onset. Stage-misassignment rate in the heat-pulse experiment must fall, with no loss in symmetric best Dice on held-out CVPPP data.
Why not just…?
Why not just average three citizens, which already matched experts? For trajectory comparisons between genotypes this is enough, as the observer study showed. The concern is that the same study found non-experts miss small leaves regardless of resolution, so their average is shifted late rather than merely noisy. The measurement component tests this directly. If citizen-consensus onset lag falls within expert-to-expert spread, the core claim fails, and this note should be read as having predicted that outcome as possible.
Why not just use temporal hindsight as post-processing? This is the real threat. Offline, tracking organs backward from later frames may recover onset as well as any rater model. Hindsight is used here as the reference for exactly that reason. The proposal's claim is restricted to the online setting, where future frames are unavailable at the moment a stress decision is needed. A ChronoRoot-style trailing average with backward fill, run with a delay of $k$ frames, is included as a baseline. If a delay of a few hours matches the rater model, the practical conclusion is "wait and smooth," and the experiment will say so.
Why not just train on expert labels only? Experts also disagree, and expert time is the scarce resource. The study will report onset error versus annotation cost for experts only, citizens only, and the rater model on mixed labels. That shows whether the model lets cheap labels substitute for expensive ones.
Why not just skip organs and use random forests on vegetation indices? For field yield, random forests beat recurrent networks by 0.5 Mg/ha in RMSE (Prediction of Spatial Winter Wheat Yield by Combining Multiscale Time Series of Vegetation and Meteorological Indices). In FeTA 2024, a model using gestational age alone reached 9.56% biometry error, which most segmentation pipelines failed to beat (FeTA 2024). In the same spirit, a random forest on plant age plus whole-plant greenness will be fit to predict stage at stress. If it matches the segmentation-based assignment, organ-level labels are not worth their cost for this question.
Decisive Checks
The idea stands or falls on three results. First, omission must dominate disagreement near onset and decay with organ size. Second, consensus-trained models must lag hindsight onset by more than inter-annotator spread. Third, in the online setting, the stage-dependent model must cut that lag and the stage-misassignment rate relative to static Pionono, delayed smoothing and the age-only forest. If the first result fails, the project reduces to a useful negative finding that extends the observer study's reassurance from counts to outlines and onset timing.
References
- Citizen crowds and experts: observer variability in image-based plant phenotyping
- Developmentally dependent reprogramming of the Arabidopsis floral transcriptome under sufficient and limited water availability
- Probabilistic Modeling of Inter- and Intra-observer Variability in Medical Image Segmentation
- The Latent Doctor Model for Modeling Inter-Observer Variability
- Inter-Rater Uncertainty Quantification in Medical Image Segmentation via Rater-Specific Bayesian Neural Networks
- Calibrate the Inter-Observer Segmentation Uncertainty via Diagnosis-First Principle
- ChronoRoot 2.0: An Open AI-Powered Platform for 2D Temporal Plant Phenotyping
- GMT: Guided Mask Transformer for Leaf Instance Segmentation
- Food security in climatic extremes: Challenges and opportunities for China
- Short-term physiological responses to drought stress in seedling of tropical and temperate maize (Zea mays L.) cultivars
- Physio-biochemical and dynamic transcriptome comparison of heat tolerance in wild and cultivated tomato accessions
- Prediction of Spatial Winter Wheat Yield by Combining Multiscale Time Series of Vegetation and Meteorological Indices
- Advances in automated fetal brain MRI segmentation and biometry: Insights from the FeTA 2024 challenge