Most medical imaging classifiers are trained on yes/no labels such as "edema present" or "edema absent," although clinicians act on finer distinctions: mild versus severe, worsening versus resolving. A binary model gives a doctor no guidance on dose, urgency, or follow-up. This proposal tests a specific bet. A training set with only coarse labels is fit equally well by many different networks. These networks should agree on textbook cases and diverge on the mild, early, or resolving ones. If we deliberately train a population of networks to spread out across this set of equally good solutions, without letting any of them get worse at the binary task, then how much the population agrees on a new image should become a graded severity signal that nobody labeled.
The mechanism comes from quality-diversity optimization, the family of methods that look for many solutions that are each near-optimal and collectively different, rather than one best solution. The application regime comes from diagnostic granularity, the gap between the binary labels models are trained on and the staged, subtyped labels clinicians need. The claim is narrow and falsifiable. On a chest X-ray benchmark that splits positives into typical and atypical cases, a population trained under a "harmless diversity" constraint should rank cases by severity better than a single model with a severity-aware loss, and better than an ordinary ensemble of the same size. It should do so with the same binary accuracy and no severity labels.
Core Claim
More Than Positive and Negative motivates the proposal. It identifies a "hard sample fallacy": cross-entropy treats atypical positives (trace edema, resolving consolidation) as hard examples and drives the model to fit them with extreme confidence, which flattens the severity spectrum inside the model. Its remedy is a single model trained with a bounded loss, Partially Huberised Cross-Entropy, inside a risk-modulation scheme (PU-RM). On a fine-grained MIMIC-CXR split this raised $AUC^{FG}$, which measures how well scores separate typical from atypical positives, from 0.7716 to 0.8038 for consolidation and from 0.6374 to 0.6663 for edema. The fix reshapes one model's loss. It never asks what the full set of equally accurate models believes about an atypical case.
How to Fill the Optimum Set? supplies the method. It trains a population of networks that repel each other while a constraint keeps their total loss from rising at every step. The paper calls this "harmless diversity." It reports that this approach beats simply adding a weighted diversity penalty to the loss. On CIFAR-10, a three-ResNet ensemble trained this way improved expected calibration error from 4.03 to 3.38 at unchanged accuracy. That paper never studies label coarseness, medicine, or ordinal structure.
Our claim connects the two. Repulsion constrained by binary loss can only move members apart along directions where the binary loss is flat. Atypical positives should sit in exactly those directions, because the coarse label does not pin down how confidently they must be scored. The population's spread on a case should therefore track severity.
Research Program
We train a population of $M$ networks with parameters $\theta_1,\dots,\theta_M$ on binary labels with the $F_{\text{sum}}$-descent rule from the optimum-set paper. Each step chooses the update that most decreases a repulsion energy $E$, subject to the summed binary loss not increasing:
$$\min_{\Delta\theta}\; \nabla E \cdot \Delta\theta \quad \text{s.t.} \quad \sum_{i=1}^{M} \nabla L(\theta_i)\cdot\Delta\theta_i \le -\epsilon \sum_{i=1}^{M}\|\nabla L(\theta_i)\|^2 .$$
Here $L$ is the binary diagnostic loss, $\Delta\theta_i$ is member $i$'s update, and $\epsilon>0$ forces steady descent. $E$ is the Newtonian Riesz energy, a sum of inverse distances between members, which that paper found better than variance because it prevents sub-clusters. We measure $E$ in function space: the distance between two members is the distance between their logit vectors on a fixed "repulsion pool" of unlabeled MIMIC-CXR images that are disjoint from training and test. This addresses the paper's stated weakness that the diversity measure must be differentiable, and it avoids needing a CLIP-style feature extractor.
The graded score is pre-registered. For a new image it is the mean member probability. The secondary score is the fraction of members that call the image positive. Both come from a binary-trained system and use no severity label.
We run three analyses. The first asks whether constrained repulsion actually concentrates disagreement on atypical positives, rather than spreading it uniformly or onto negatives. The second asks whether the graded score transfers to a second modality with true ordinal labels. The third asks whether the effect survives when each member is itself trained with PCE, which would show the two remedies are complementary.
Baselines
The naive reference point is one DenseNet-style classifier trained with standard cross-entropy on binary labels, using the U-Ones policy for uncertain report labels. Its output probability serves as the severity score. This is the U-Ones baseline in the fine-grained paper, with reported $AUC^{FG}$ of 0.7716 for consolidation. For edema, the file reports a baseline of 0.6374 without naming the policy. We rerun it on our splits to control for implementation drift.
The strongest existing single-model method is PU-RM with PCE loss from More Than Positive and Negative, at 0.8038 for consolidation and 0.6663 for edema. It is the most direct competitor because it targets the same failure with the same supervision.
The strongest ensemble competitor is a standard deep ensemble of $M$ members from different random seeds. Its spread is the cheap version of our signal. Every ensemble uses the same $M$ (five), backbone, epochs, augmentation, and number of hyperparameter trials, so the comparison is between methods rather than compute budgets.
Two ablations isolate the key ingredient, which is the hard non-increase constraint. The first replaces it with a weighted penalty, binary loss minus $\lambda E$, with $\lambda$ swept over the same trial budget. The second keeps the constraint but measures repulsion in weight space instead of function space.
Beating the field means three things on the MIMIC-CXR fine-grained split. $AUC^{FG}$ must exceed both 0.8038 (consolidation) and 0.6663 (edema). It must exceed the equal-size deep ensemble by a margin larger than the seed-to-seed standard deviation across three population replicates. Binary AUROC must not drop by more than half a point relative to the deep ensemble.
Why not just…?
Why not just use the disagreement of an ordinary deep ensemble? This is the real threat. Random-seed ensembles already disagree off the training set, and the file contains no medical result comparing them with harmless-diversity populations. The CIFAR-10 calibration gain from 4.03 to 3.38 suggests that constrained repulsion spreads members more usefully than initialization alone. Calibration on natural images is not severity ranking on radiographs, though. The equal-size deep ensemble is therefore a headline comparison, not a straw man. If it matches us, the honest conclusion is that ensemble disagreement recovers granularity and the constraint adds nothing, and we will report that.
Why not just add a diversity penalty to the loss? The optimum-set paper shows that a weighted penalty trades accuracy for diversity, while the constrained update moves the trade-off curve itself. In medicine this matters because a penalty can buy diversity by making some members worse at the binary call. The lambda-sweep ablation tests this directly. The prediction is that, at matched binary AUROC, the penalized ensemble has lower $AUC^{FG}$.
Why not just use PU-RM's bounded loss and stop there? PU-RM already addresses the hard sample fallacy in one model, and it may capture most of the available signal. We test this two ways: against PU-RM directly, and against a population whose members each use PCE. If the PCE population beats single-model PU-RM, the two effects stack. If it does not, population diversity is redundant with loss bounding.
Why not just train on severity labels where they exist? Where expert staging exists, supervised training is the right answer. The neonatal echocardiography work trains a three-class severity model directly. The proposal's point is that such labels are scarce and costly, while binary labels are abundant. The echocardiography experiment below measures how much of the supervised severity performance a binary-trained population recovers.
Proposed Experiments
The primary benchmark is the fine-grained split of MIMIC-CXR defined in More Than Positive and Negative. It divides consolidation and edema positives into typical (severe, acute, worsening) and atypical (mild, trace, improving) cases using report text. We report $AUC^{FG}$ and binary AUROC for every system.
We also report atypical-positive-versus-negative AUROC. This guards against a degenerate outcome where the population is simply uncertain about everything hard, including ambiguous negatives. Without this metric, such a result could look like severity sensitivity.
The decisive diagnostic is a disagreement profile. We plot the population's mean pairwise logit spread separately for negatives, atypical positives, and typical positives. The hypothesis predicts the spread is highest for atypical positives and higher under the constrained population than under the seed ensemble. If the spread is flat across groups, or tracks proxies for image acquisition rather than severity, the mechanism is wrong even if $AUC^{FG}$ moves. We will check this by correlating spread with portable-versus-standard view metadata where available.
The transfer test uses the neonatal echocardiogram cohort of Ragnarsdóttir et al.: 1,311 videos from 270 newborns, with a ResNet3D-18 backbone. We train populations on binary pulmonary-hypertension labels only. We then measure Spearman correlation and ordinal AUROC between the graded score and the cardiologist's None / Mild / Moderate-to-Severe labels on the held-out test set. The reference is that paper's supervised severity model, at F1 0.84 in cross-validation and 0.63 on held-out test, and its binary model, at 0.92 and 0.78. The file reports no ordinal-correlation number for any binary-trained model on this cohort, so the comparison point is a supervised three-class model we retrain and threshold. The cohort is small, so we use patient-level bootstrap intervals and treat this as a secondary check.
We consider the proposal confirmed only if three conditions hold together. The constrained population must beat PU-RM and the equal-size deep ensemble on $AUC^{FG}$ for both findings. The disagreement profile must peak on atypical positives. The penalty ablation must lose at matched binary accuracy. Any one failing narrows the claim, and we will say which.
References
- More Than Positive and Negative: Communicating Fine Granularity in Medical Diagnosis
- How to Fill the Optimum Set? Population Gradient Descent with Harmless Diversity
- MIMIC-CXR: A large publicly available database of labeled chest radiographs
- Deep Learning Based Prediction of Pulmonary Hypertension in Newborns Using Echocardiograms