Diabetic retinopathy is graded on a five-step scale (none, mild, moderate, severe, proliferative), and the scale is defined by which lesions are visible, not by an overall impression of the image. "Mild" means microaneurysms only. Microaneurysms are tiny bulges in retinal capillaries, often only 10–20 pixels across. "Moderate" means something beyond microaneurysms, such as hemorrhages or exudates, and "severe" depends on how many lesions appear in each quadrant of the retina. Every routine image-level grade is therefore a weak but exact statement about the lesions the image must or must not contain. A grade-0 image says "there are zero lesions here." A grade-1 image says "there are microaneurysms and nothing else." We propose to train a small-lesion detector using only these grade-implied constraints, with no boxes or masks, and to force the final grade to be computed from the detected lesions through a near-symbolic rule. The resulting system should localize the evidence for its grade and should keep its accuracy when the fundus camera changes. The clinical rule does not depend on the camera, even though global image statistics do.
This targets a specific failure in the current literature. Methods that discover lesions from grades alone were evaluated only within one data distribution. Methods that generalize across cameras with explicit lesion evidence buy that evidence from fully supervised detectors. Methods that study camera shift collapse the five grades into a binary referral decision. Our bet is that the ordinal structure of the grade, the dimension usually treated as a nuisance for loss design, carries enough information to supervise tiny-object attention, and that a grade computed from counted lesions is a better route to robustness under shift than a grade read off a global embedding.
Core Claim
A detector trained only on routine five-level grades, under losses that encode what each grade implies about lesion presence and type, can recover small lesions well enough that a grade computed from its lesion counts will (a) match the in-domain accuracy of strong global classifiers and (b) degrade substantially less when trained on one camera population and tested on others. The two ingredients that matter are the grade-to-lesion constraints, especially the large supply of grade-0 images, which give free negatives for every pixel, and the counting bottleneck between lesions and grade. High-resolution, size-balanced attention is necessary so that microaneurysms survive into the counting step, but it is supporting machinery rather than the claim.
Relation to Prior Work
The closest work is Lesion-Aware Transformers for Diabetic Retinopathy Grading. It learns lesion queries in a cross-attention decoder from image-level grades and adds a diversity loss so that different queries look at different places. It motivates our proposal because it shows lesions can emerge from grade supervision. It differs in three ways. Its grade is still predicted from pooled features rather than from lesion counts. It never uses the fact that a grade forbids certain lesions. And its summary in the file reports no cross-camera evaluation. Its decoder also operates at standard resolution, where the smallest microaneurysms risk being diluted.
Single Domain Generalization in Diabetic Retinopathy: A Neuro-Symbolic Learning Approach (KG-DG) is the nearest work on robustness. It feeds lesion detections from a supervised YOLOv11 and vessel maps from a U-Net into symbolic rules beside a vision transformer, and reports a 5.2% accuracy gain across APTOS, EyePACS, Messidor-1 and Messidor-2. KG-DG motivates the counting bottleneck because lesion evidence helped it generalize. It also constrains our design, because its vessel branch hurt cross-domain accuracy: fine anatomy extracted by a camera-sensitive model became the weak point. We therefore use only lesion evidence, learn it without dense labels, and test whether weakly learned lesions avoid the same fragility.
Domain Adaptation for Ulcerative Colitis Severity Estimation Using Patient-Level Diagnoses supplies the logic we reuse. It treats a patient's maximum-severity report as an upper bound on every frame in the patient's set of images, penalizes frames predicted to exceed it, and reaches 71.4% cross-hospital accuracy against 52.1% for standard unsupervised adaptation. We move that bounding idea one level down, from "frame severity is at most patient severity" to "lesion types present are fixed by image grade," and add spatial localization, which that work lacks.
Two papers supply the small-object machinery. Pseudo-Labeling for Small Lesion Detection on Diabetic Retinopathy Images shows that microaneurysms need roughly 2000×2000 inputs and a six-level feature pyramid, meaning six stacked feature maps at decreasing resolution, to survive downsampling. That work also bootstraps from partial boxes, which we do not have. Small Objects Matters in Weakly-supervised Semantic Segmentation shows that tag-supervised localization ignores small instances unless losses are reweighted by size, with up to 8.9 points of small-object gain on PASCAL-B. We adopt its instance-aware evaluation idea and its size balancing.
Finally, Avoiding Catastrophic Referral Failures In Medical Images Under Domain Shift shows that even Bayesian models become confidently wrong when moving from EyePACS to APTOS, and that post-hoc recalibration helps. Its analysis is binary, which leaves open whether five-level staging can be protected at the feature level.
Research Program
The model has three parts. The first is a high-resolution backbone with a six-level feature pyramid in the style of Chen et al. The second is a lesion head that applies lightweight spatial attention on the two finest pyramid levels and outputs dense per-pixel "lesion density" maps for a small number of lesion channels. Summing a map over a region gives a soft count of lesions there. The third is a grading head that sees only these counts: per-channel totals and per-quadrant totals. It maps them to an ordinal grade through a monotone function with a handful of learned thresholds, so that more lesions can never lower the grade.
Lesion channels acquire identity from the grading ladder rather than from labels. The microaneurysm channel is defined as the only channel allowed to be non-zero in grade-1 images. The "beyond-microaneurysm" channel or channels are those required to be non-zero at grade 2 and above. Writing $c_m(x)$ for the soft microaneurysm count and $c_o(x)$ for the soft count of other lesions in image $x$ with grade $y$, the constraint loss is
$$ \mathcal{L}_{\text{rule}} = [y{=}0]\,(c_m + c_o) + [y{=}1]\,\big(c_o + \max(0, 1 - c_m)\big) + [y{\ge}2]\,\max(0, 1 - c_o), $$
where $[\cdot]$ is 1 when the condition holds and 0 otherwise. The terms enforce "nothing," "microaneurysms only," and "something more," respectively. We add an ordinal loss on the rule head's output and the size-balanced weighting of Mun et al. on the density maps. The size weighting keeps a single correctly placed microaneurysm from being drowned out by large exudates.
Grade labels in EyePACS are noisy, so all constraints are hinges with a tolerance, and we report sensitivity to the tolerance. We are explicit about one limit. The ladder separates microaneurysms from everything else, but it cannot separate hemorrhages from exudates, so we expect at most two reliably named channels. The quadrant-count criterion for "severe" can be represented, but only coarsely.
We study two regimes. In single-source generalization, we train on EyePACS grades and test unchanged on APTOS, Messidor-2 and DDR. In grade-only adaptation, a target site provides a small number of routine grades and we fine-tune with the same constraint loss. This mirrors the patient-level setting of Yamaguchi et al.
Lesion quality is measured on DDR's lesion annotations, which are never used for training. We report instance-level recall of microaneurysms and other lesions at fixed false positives per image, scored with a center-hit criterion rather than box overlap, because Chen et al. show box overlap unfairly punishes tiny targets.
Baselines
The naive reference is a standard high-resolution convolutional classifier trained with cross-entropy on EyePACS grades. It uses the same backbone, input size, augmentation and epoch budget as our model and is evaluated on the same targets. Any claim about robustness must first beat its cross-camera drop.
The strongest in-domain grader in the file is MAFNet. It combines multi-scale and relational attention with a joint classification-plus-regression loss, and reports QWK 0.934 on DDR, 0.936 on APTOS and 0.917 on Messidor-2. QWK, quadratic weighted kappa, is agreement with graders that penalizes errors by the square of their grade distance. These numbers come from in-domain training. We will retrain MAFNet on EyePACS only and evaluate it cross-camera, where no reported result was found on Lacuna. Beating it means matching it within 0.01 QWK in-domain while showing a smaller cross-domain QWK drop, averaged over the three targets with confidence intervals across three seeds.
The strongest weakly supervised lesion-discovery competitor is the Lesion-Aware Transformer, which reports QWK 0.884 on EyePACS and referral AUC 0.987 on Messidor-1. No cross-domain result for it was found on Lacuna. We run it at our input resolution and at its native resolution, with the same tuning budget of an equal number of hyperparameter trials. Beating it requires both higher cross-camera QWK and higher small-lesion recall on DDR.
The strongest robustness competitor is KG-DG, run with its supervised lesion detector. This comparison deliberately grants it lesion annotations we do not use. Its reported figure is the 5.2% average cross-domain accuracy gain; per-dataset QWK for it was not found on Lacuna. Matching KG-DG without any lesion labels would count as success.
For referral safety, we compare against Split Platt Scaling from Band et al. applied to the naive baseline, at matched referral rates.
The key ablations isolate the ingredients. The first keeps every module but replaces the count-only grading head with an MLP on pooled features, which removes the bottleneck. The second drops the grade-0 term of the constraint loss. The third removes size balancing. The fourth replaces the six-level pyramid with a standard four-level one.
Why not just…?
Why not just fine-tune a retinal foundation model at high resolution with an ordinal head? This is the most serious threat. Parameterized Diffusion Optimization Enabled Autoregressive Ordinal Regression for Diabetic Retinopathy Grading reaches 80.3% APTOS accuracy with a RETFound backbone. However, nothing in the file shows such models staying accurate across cameras for five-level grading, and SpurCon shows foundation features can collapse on artifacts. We will include a RETFound backbone fine-tuned under the same protocol. If it matches our cross-camera QWK, the counting bottleneck is not buying robustness. In that case we would report the contribution as label-free localization only, and we would also put the constraint loss on top of RETFound to see whether the two add.
Why not just use the Lesion-Aware Transformer with heavy color and blur augmentation? Augmentation targets the illumination and focus changes that dominate fundus shift, and it could close much of the gap. We will run the Lesion-Aware Transformer and the naive baseline with the same augmentation policy as our model. All methods receive it, so any remaining difference is attributable to the constraints and the bottleneck.
Why not just train a supervised lesion detector on DDR boxes and count its outputs? This is the KG-DG route and is cheap once a lesion dataset exists. The vessel failure in KG-DG suggests supervised fine-anatomy extractors can be brittle across cameras, but lesion detectors helped there, so this is a live alternative. We test it directly: a DDR-trained detector feeds the same rule head, and we evaluate on APTOS and Messidor-2. If it beats our weak detector cross-camera, the honest conclusion is that grade constraints are a cheaper substitute rather than a better one, and we would measure how many boxes are needed to break even.
Why not just recalibrate outputs post hoc at each site? Band et al. show recalibration fixes binary referral thresholds. Recalibration cannot restore microaneurysms lost in early pooling, and it says nothing about mild-versus-moderate confusion. We compare against it at equal referral rates and also report five-level QWK, where post-hoc logit scaling cannot change the ranking much.
Proposed Experiments and Decisive Checks
The primary result is the EyePACS-to-{APTOS, Messidor-2, DDR} QWK drop for every method above. Secondary metrics are macro F1 for the rare severe and proliferative grades, and referral sensitivity at fixed specificity.
The first decisive check is the bottleneck ablation. If the pooled-feature head generalizes as well as the count-only head, the central robustness claim fails.
The second check is lesion recall on DDR, broken out by size. If microaneurysm recall is near chance, the grade-1 constraint is not doing its job. In that case any robustness gain is coming from elsewhere and must be reinterpreted.
The third check probes the biomarker paradox: we measure whether our lesion counts shift less across cameras than the counts from the supervised DDR detector. We do this by comparing count distributions within each grade across sites.
The fourth check, in the grade-only adaptation regime, asks whether constraint-based fine-tuning on 100 to 500 target grades beats plain classifier fine-tuning on the same grades. This would replicate, at lesion level, the advantage Yamaguchi et al. found over semi-supervised fine-tuning.
We expect gains to be largest at the mild-versus-none boundary, because that boundary depends entirely on detecting tiny lesions. We expect gains to be smallest for proliferative disease, which the ladder constrains least.
References
- Lesion-Aware Transformers for Diabetic Retinopathy Grading
- Single Domain Generalization in Diabetic Retinopathy: A Neuro-Symbolic Learning Approach
- Domain Adaptation for Ulcerative Colitis Severity Estimation Using Patient-Level Diagnoses
- Pseudo-Labeling for Small Lesion Detection on Diabetic Retinopathy Images
- Small Objects Matters in Weakly-supervised Semantic Segmentation
- Avoiding Catastrophic Referral Failures In Medical Images Under Domain Shift
- MAFNet: A novel adaptive multi-scale model for fine-grained grading of diabetic retinopathy
- Parameterized Diffusion Optimization Enabled Autoregressive Ordinal Regression for Diabetic Retinopathy Grading