Counting Layers With Many Hands: Rater-Aware Age Uncertainty for Laminated Records

Many records of Earth's past, including annually laminated lake sediments, marine cores and banded deposits, are dated by counting layers. An expert looks at an image of the core and marks where each layer ends. Different experts mark different boundaries, and the same expert marks different boundaries on different days. In medical imaging this disagreement is studied as inter-observer variability, and there are good models for it. In a layered record, though, a disputed boundary is not a local error. Every missed or extra boundary shifts the age of everything below it, so small disagreements add up along the core into large errors in time.

This note proposes a model that learns how each annotator tends to err, samples whole sequences of layer boundaries that respect depth order, and turns those samples into age-depth curves. It then adjusts how much it trusts each annotator using a few independent radiometric dates, and it is built to tolerate some of those dates being wrong. The central claim is a testable one. Annotator errors are partly systematic: a given person may habitually split faint sub-annual laminae, or habitually merge them. When errors are systematic, the uncertainty in age grows roughly with the square of depth rather than linearly. Standard layer-counting error budgets, and generic uncertainty methods such as model ensembles, treat errors as independent and should therefore understate deep-core age uncertainty. A model with an explicit per-annotator latent style should not.

Core Claim

Suppose the true age at layer $n$ is approximated by a count. If each annotator's per-boundary errors were independent with variance $\sigma^2$, the variance of the count error would be $n\sigma^2$. If annotator $r$ also carries a systematic split-or-merge rate $b_r$, drawn across annotators with variance $\tau^2$, the error at layer $n$ is approximately $n b_r$ plus independent noise. That gives

$$\mathrm{Var}[\text{age error at layer } n] \approx n\sigma^2 + n^2\tau^2,$$

where the second term comes from the same bias repeating at every boundary. The proposal has two parts.

The first is a probabilistic boundary model in which each annotator is a distribution in a small latent space. This follows Pionono (Schmidt et al.), which represents each annotator as a Gaussian whose mean captures systematic bias and whose variance captures that person's inconsistency. Unlike Pionono, our decoder outputs an ordered sequence of boundaries. We do not output an unordered mask.

The second is a "chronology-first" calibration step. It reweights annotator latents so that the resulting age-depth curves agree with sparse radiometric tie-points, meaning independent dated depths. The fit uses a heavy-tailed likelihood, so a corrupted date is down-weighted rather than allowed to drag the whole chronology.

What is new is the combination of ordered, depth-monotone outputs, annotator style that drives errors which accumulate along the core, and a downstream target that is continuous, sparse and itself noisy. None of the multi-annotator work in the file addresses this setting, and nothing in the file models annotator disagreement in geological records at all.

Relation to Prior Work

The motivation comes from Direction 1. Matsubara shows that one systematic interpretive error, misidentifying index fossils, falsely synchronized marine events and shifted a regional chronology by about 20 million years. That is the cascading, correlated failure our quadratic term describes, here at the scale of a whole formation. Sediment-core studies on the Songnen Plain (Chen et al.), in Laizhou Bay (Liu et al.) and in deglacial permafrost records (Yan et al.) build environmental histories on depth-to-time conversions. None of them quantifies how much of their timing uncertainty comes from human interpretation of the record.

The in situ dating papers (Wang et al.; Hou et al.) date hydrothermal minerals directly and so avoid assumptions borrowed from the host rock. Their lesson limits our scope. A measured signal can be decoupled from the event it is supposed to date, so our tie-points must be treated as possibly wrong. We do not address mineral-rim delineation.

The method comes from Direction 2. The variational model of Liu et al. learns one shared latent distribution over annotator disagreement. It reports a continuous Dice of 0.5372 on QUBIQ brain growth, against 0.5083 for MC dropout. It does not separate annotators, so it cannot represent one annotator's bias repeating along a core. Pionono does separate annotators and reaches an unweighted Cohen's κ of 0.758 on Gleason 2019, but it assumes spatially unordered labels. DiFF (Wu et al.) weights annotators by how much they help a downstream diagnostic classifier, with AUC gains of up to 2.73% on REFUGE-2. Its downstream truth is a clean discrete label, whereas ours is a few noisy continuous dates. Its weights are also tied to one classifier architecture. Our weights are instead tied to a physical age model, which removes that dependence. The QUBIQ organizers (Li et al.) found that plain deep ensembles often match complex probabilistic models, with average Dice of 0.80 to 0.85. That finding constrains us: we must beat ensembles on age calibration specifically, not on segmentation overlap.

Research Program

We will build a boundary detector that works on 2D core images, so that it can handle tilted beds, cracks and inclusions. It is conditioned on an annotator latent $z_r$. Its output is a sequence of boundary depths, decoded under a constraint that depths increase down the core. One way to do this is to predict positive layer thicknesses and take a cumulative sum.

At training time, each annotator's marked boundaries supervise the decoder under that annotator's latent. At test time on a new core, we sample $z$ from the population of annotator latents, decode boundary sequences, and convert each one to an age-depth curve. This yields a full distribution over ages at every depth.

The chronology-first step then fits mixture weights over annotator latents, plus a slowly varying correction to the accumulation rate. The fit maximizes a Student-t likelihood of the tie-point ages under the sampled curves, using only dates that are not held out. Analytically, we will estimate $\sigma^2$ and $\tau^2$ from the data and check whether the quadratic term is real and large enough to matter.

The data has to be created, because the file contains no multi-annotator geological benchmark. It will come in two parts. The first is a synthetic generator for laminated cores whose true counts are known. It will include faint years, sub-annual laminae, tilt, cracks, and thick event layers from high-energy flows, which should not be counted. Simulated annotators will have controlled values of $b_r$ and noise. The second part is a real annotation campaign. Five to eight annotators will independently mark boundaries on laminated core images that have independent radiometric dates. Each annotator will re-mark a subset after a delay, so that we can measure each person's day-to-day inconsistency. On real cores no reported result was found on Lacuna for any multi-annotator layer-counting method, so the baselines below set the bar.

Baselines

The do-nothing reference is the common practice of taking one expert's count, attaching a fixed fractional error, and interpolating between radiometric dates.

The strongest naive multi-annotator reference is the empirical annotator spread. We build one age-depth curve per annotator and report the range between them. This is only available on cores where every annotator has done the marking.

The strongest generic uncertainty method is a deep ensemble of deterministic boundary detectors trained on majority-vote boundaries, as favored by Li et al..

The strongest existing annotator-aware method is Pionono (Schmidt et al.), adapted to 1D boundary heatmaps with boundaries read off by peak picking. The obvious calibration competitor is DiFF-style weighting (Wu et al.), with age error at tie-points as the downstream loss and a Gaussian likelihood.

To check that our Pionono re-implementation is faithful, we will first reproduce its κ = 0.758 on Gleason 2019 before porting it.

All learned methods will use the same backbone, the same training cores, the same number of tuning trials and the same GPU budget. For ensembles we will report both an equal-compute version and a five-member version.

Ablations will remove one ingredient at a time:

  • per-annotator latents, replaced by a single shared latent in the style of Liu et al.;
  • the depth-order constraint;
  • tie-point calibration;
  • the heavy-tailed likelihood.

To count as beating the baselines, our method must give better-calibrated age intervals at held-out dates in the deep half of cores. Concretely, empirical coverage of the nominal 90% interval must be within five points of 90% while the ensemble and DiFF-style methods undercover. Our intervals must also not be wider than the empirical annotator spread on cores where that spread exists.

Why not just…?

Why not just have several annotators count each core and report the spread? On fully annotated cores this is a real threat, and it is close to the ground truth for annotator variation. Its weaknesses are cost and reach. It needs every annotator on every core and gives nothing for new cores, and with five to eight annotators its tails are poorly estimated. We will treat it as a reference: our model's intervals on fully annotated held-out cores should match its spread. If our model only matches it and adds nothing on single-annotator cores, the contribution narrows to cheaper extrapolation, and we will say so.

Why not just use a deep ensemble or MC dropout? These methods capture the model's uncertainty about its own parameters, not the systematic styles of the people who made the labels. Liu et al. found MC dropout trailing a latent annotator model on QUBIQ, though Li et al. found ensembles competitive on overlap. Our prediction is specific: ensembles will look fine on boundary F1 but undercover age at depth, because their errors are nearly independent across layers. The synthetic sweep over $\tau$ settles this directly.

Why not just trust radiometric dates and interpolate? Dates are sparse and can be corrupted, for example by incomplete bleaching of luminescence signals after rapid deposition. The decoupling results of Wang et al. and Hou et al. show that a measured age can reflect a different event from the one intended. Leaving out dates and checking the interpolation-only baseline against held-out dates will show how much the layer counts add between tie-points.

Why not just apply a 1D peak detector to a grayscale scan line? This is cheap and, on clean horizontal varves, possibly adequate. We will include it as a baseline. We expect it to fail on the tilted, cracked and event-layer synthetic cores, and an ablation by difficulty stratum will show where it breaks.

Decisive Experiments

On synthetic cores, we vary $\tau$ from zero upward. The prediction is that independent-error baselines stay calibrated only at $\tau=0$ and undercover increasingly with depth as $\tau$ grows, while our model tracks the true $n^2\tau^2$ growth. If observed coverage does not degrade with $\tau$ for the baselines, the core claim fails.

On real cores, we estimate $\tau^2$ from the annotation campaign. If it is negligible relative to $\sigma^2$, the per-annotator machinery is unnecessary and the simpler shared-latent model suffices. We will report that outcome as a finding.

To test robustness, we deliberately corrupt one tie-point per core. The heavy-tailed calibration should keep held-out coverage near nominal, while Gaussian DiFF-style weighting should be pulled toward the corrupted date.

Metrics are:

  • count error and boundary F1 within a depth tolerance;
  • absolute age error at held-out dates;
  • the continuous ranked probability score of the predicted age distribution, which rewards both accuracy and honest spread;
  • interval coverage reported separately for the upper and lower halves of each core.