Rater Thresholds on a Recovered Optic Cup Surface

Different eye specialists who outline the optic cup on the same retinal photograph draw different boundaries. The optic cup is the pale central depression of the optic nerve head, and its boundary drives the cup-to-disc ratio used to screen for glaucoma. Current models of this disagreement treat it as abstract noise in the 2D image: each annotator gets a learned latent vector, and a network decodes that vector into a mask. We propose a more physical explanation. The cup is a 3D bowl seen from above in a single photograph. Its surface can be partly recovered from shading. Each annotator can then be modeled as choosing where on that bowl to stop, using a depth level and a relative weight on paleness versus depth. If this is right, disagreement should be largest exactly where the bowl's walls are shallow. That prediction can be checked on the seven-specialist annotations of REFUGE (REFUGE Challenge) before any rater model is trained.

The recipe has three parts. First, we recover a height field and a reflectance (albedo) map from the fundus image with a low-order spherical harmonic lighting model. Spherical harmonics are a smooth basis over directions, and nine coefficients per color channel describe diffuse illumination well. Second, we model each rater's mask as the level set of a combination of recovered depth and paleness, with two or three interpretable parameters per rater. Third, we train through the hard "inside or outside the level set" decision. We do this by borrowing the rasterize-then-splat trick: find the boundary with a non-differentiable search, then compute exact gradients only at the boundary points found. If the approach works, it gives a rater model that separates lighting and camera effects from genuine clinical judgment. That separation matters when a model trained on one fundus camera is deployed on another.

Core Claim

Much of the inter-observer disagreement on the optic cup can be explained by a shared 3D surface plus a few per-rater parameters. The model is

$$M_r(x) = H\big(\alpha_r\, d(x) + (1-\alpha_r)\, p(x) - t_r\big),$$

where $M_r(x)\in\{0,1\}$ says whether rater $r$ places pixel $x$ inside the cup. $H$ is the Heaviside step function, $d(x)$ is recovered depth below the disc rim, and $p(x)$ is recovered pallor, meaning albedo brightness after shading has been removed. The per-rater parameters are a depth-versus-pallor weight $\alpha_r$ and a threshold $t_r$. An intra-rater Gaussian spread on $t_r$ captures session-to-session inconsistency. Writing $f = \alpha_r d + (1-\alpha_r) p$ for the combined score, a first-order expansion of the level set gives a direct prediction. A threshold perturbation $\delta t$ moves the boundary by

$$\delta x_\perp \approx \frac{\delta t}{\lVert \nabla f(x) \rVert},$$

so boundary spread across raters should scale with the inverse steepness of the recovered surface. This physical, falsifiable claim is what separates the proposal from latent-style models. Pionono gives each rater an 8-dimensional Gaussian concatenated to U-Net features. The Latent Doctor Model builds a clinician style embedding from support examples. OMBA-UNet uses one Bayesian decoder per rater. All three can fit rater styles, but none says why a given image or boundary segment is contested. None can separate a lighting change from a change in annotator behavior.

Research Program

The first component is photometric decomposition of the optic nerve head. Following SfSNet, which decomposes face images into normals, albedo and 27 second-order spherical harmonic lighting coefficients through a differentiable Lambertian layer, we render

$$\hat I(x) = A(x) \odot \big(\mathbf{Y}(\mathbf{n}(x)) \cdot \mathbf{l}\big).$$

Here $A$ is albedo, $\mathbf{n}$ are normals derived from the height field, $\mathbf{Y}$ stacks the nine second-order spherical harmonic basis functions, and $\mathbf{l}$ holds per-image lighting coefficients. The network is trained self-supervised on photometric reconstruction of the disc crop. The retina is not Lambertian: the inner retinal surface produces specular glints, and SfSNet itself reports failures on non-Lambertian skin and cast shadows. We therefore mask saturated glints from the photometric loss and add a smoothness prior on the height field.

We do not claim metric depth. The recovered surface only has to be ordinal and locally reliable enough that its gradient is meaningful. Scale is absorbed into $t_r$. The collapse of lighting integrals into dot products, exploited for relighting by Kaleta et al. and Ikeda et al., is what keeps this layer cheap.

The second component is the rater model above. It is fit jointly with the decomposition on all seven REFUGE annotations. The decomposition network sees the image, so lighting and camera differences can move $\mathbf{l}$ and $A$, but never rater parameters.

The third component is the gradient through $H$. The obvious relaxation replaces $H$ with a sigmoid of temperature $\tau$. This is the soft-boundary idea of the Neural 3D Mesh Renderer, which blurs gradients over a margin around edges. Its cost is a biased mask for any finite $\tau$, and the bias is worst on thin rims, where Dice is most sensitive. We instead adapt Rasterize-then-Splat. That method locates visible surface points with non-differentiable z-buffering or marching cubes, then evaluates attributes differentiably at those points. For us, marching squares extracts each rater's contour from $f$ with no gradient. The contour points are then moved along the normal by the implicit-function derivative $-\nabla f/\lVert\nabla f\rVert^2$. Finally, they are splatted into a soft boundary loss against that rater's mask. The forward mask stays exact, and gradients reach $d$, $p$ and $t_r$ only through the boundary. This is the proposal's use of the non-differentiable rasterization literature. The sampling problem is the same as in RtS, applied to a 2D level set rather than a mesh.

Baselines

The naive reference is a U-Net trained on the pixelwise average of the seven masks. Its soft output is read as the rater distribution, and its predictions are thresholded for a single mask. It costs nothing to model raters. If our approach cannot beat it on distribution metrics, the rater model adds nothing.

The strongest existing distribution model is Pionono. Its per-rater Gaussian latents directly compete with our per-rater thresholds. It reports $\kappa = 0.758$ on Gleason 2019 and $\kappa = 0.711$ on a 25-rater breast cancer set. We found no result on Lacuna for Pionono, the Latent Doctor Model or OMBA-UNet on REFUGE. We will therefore rerun Pionono with its published ResNet34 U-Net and 8-dimensional latent on our splits, with the same crops, augmentation, epochs and hyperparameter search budget as our model.

For consensus cup accuracy, the reference points in the file are Masker's optic cup Dice of 0.8837 in the REFUGE challenge and RF-Net at 89.79% cup Dice on REFUGE-2018. We do not expect to beat these on consensus Dice, and that is not the claim. Matching them within about one Dice point is the bar for showing the physical bottleneck does not cost accuracy.

For downstream use, DiFF reports up to 2.73% glaucoma AUC improvement on REFUGE-2. It does this by weighting raters for diagnostic value and smoothing expertness maps with a 2D module. We will compare against DiFF using vertical cup-to-disc ratio derived from each model's calibrated mask, rather than retraining its classifier.

Two ablations isolate the key ingredients. The first keeps the level-set rater model but replaces $d$ and $p$ with an unconstrained scalar field predicted by a U-Net, with no photometric loss. This tests whether physics matters or only the threshold structure. The second swaps the contour-sampling gradient for a sigmoid relaxation at several temperatures. A variant with zeroth-order spherical harmonics (constant lighting) tests whether directional lighting matters.

Beating Pionono means one of three outcomes: equal in-distribution distribution metrics with 3 instead of 8+ parameters per rater; a clear win on the cross-camera split; or better correlation of predicted and observed per-image disagreement.

Why not just…?

Why not just use Pionono or the Latent Doctor Model, which already fit rater styles? On in-distribution REFUGE they may well match us on distribution metrics. We say so plainly, because an 8-dimensional latent is strictly more flexible than two thresholds. The claim that could separate them is under distribution shift. REFUGE was acquired on a Zeiss Visucam 500 and a Canon CR-2. A latent model trained on one camera has no mechanism to stop illumination differences from leaking into rater latents. Our model routes them into $\mathbf{l}$ and $A$. The cross-camera experiment settles this. If Pionono transfers equally well, the physical story adds interpretability but not robustness, and we will report it that way.

Why not just predict any scalar field and threshold it per rater, without shading physics? This is the most serious threat. The level-set structure alone may explain most of the benefit. The unconstrained-field ablation tests it directly. The photometric version should win specifically on cross-camera transfer, and on how well $1/\lVert\nabla f\rVert$ predicts boundary spread on held-out images. The unconstrained field is fit only to masks, so its gradient has no reason to track image evidence away from the training raters. If the ablation ties on both, we drop the lighting component and publish the simpler model.

Why not just normalize contrast (for example with CLAHE) instead of modeling lighting? Normalization changes intensities globally. It does not separate a darkened rim caused by surface slope from pallor caused by tissue. That separation is exactly what $\alpha_r$ needs. We include a CLAHE-preprocessed Pionono in the cross-camera test, so this question gets an empirical answer.

Why not just use a sigmoid relaxation of the threshold? It is simpler and may suffice. We expect a systematic, temperature-dependent bias in boundary placement on thin cup rims. The relaxation ablation measures this bias against the contour-sampling gradient at matched compute.

Proposed Experiments and Decisive Checks

The first check needs no rater training and can kill the idea early. We fit only the photometric decomposition on REFUGE training images. Along radial sectors of each disc, we then measure the standard deviation of the seven specialists' cup boundary positions and correlate it with the inverse recovered slope at the consensus boundary. The inverse-slope relation above predicts a positive correlation, and pallor edges should add to it where depth is flat. A correlation near zero means observers are not responding to recoverable geometry, and the program stops there.

The main experiments use REFUGE's seven-rater annotations. On the standard split, we report consensus cup Dice and vertical cup-to-disc ratio mean absolute error, the metric on which Masker reached 0.0414. For distribution fit, we adapt the QUBIQ Q-Dice, which averages Dice over thresholds 0.1 to 0.9 as defined in QUBIQ, together with generalized energy distance between sampled and true rater masks. We also measure per-rater held-out Dice: train on six raters' masks for some images, then predict each rater on new images. The cross-camera split trains on one device and tests on the other. We will also test few-shot new-rater fitting, holding out one specialist entirely and fitting their $(\alpha_r, t_r)$ from five images. This is the regime where the Latent Doctor Model's support-set encoder is the natural comparison.

Drishti-GS, with 101 images where E-DCoAtUNet reports 90.8% cup Dice, serves as an external consensus test with no retraining. For downstream use, we feed rater-sampled vertical cup-to-disc ratios to a fixed glaucoma classifier and compare AUC against DiFF's calibrated targets. This helps address DiFF's own acknowledged risk of overfitting its pseudo ground truth to one classifier.

The single decisive result is the cross-camera comparison combined with the slope–spread correlation. If the photometric model holds rater parameters fixed across devices while Pionono's per-rater agreement degrades, the physical account of disagreement has earned its place. If it does not, the unconstrained-field ablation tells us which simpler model to keep.