Finding the Diagnostic Granularity a Cohort Actually Supports

Many medical datasets carry only coarse labels such as normal versus abnormal or benign versus malignant, yet treatment depends on finer distinctions like tumor subtype or severity grade. We propose to recover those finer strata without fine labels. The method clusters patients within each coarse class many times, each time on a different slice of the data where a nuisance factor (scanner, magnification, site, or patient set) is held fixed. It then keeps only the sub-groups that reappear across slices. Ensemble clustering is the usual tool for stabilizing clusters. It combines many "base" clusterings into one consensus. Here it is used for a different job: to separate disease structure that replicates from visual confounders that do not.

The opportunity matters because the coarse-label bottleneck is institutional, not biological. Archives standardize on binary ground truth to control label noise, and binary flags give no therapeutic direction (Ragnarsdóttir et al.). Supervised fine-grained models need expensive annotation, and they lose much of their accuracy at new sites. The neonatal pulmonary-hypertension severity F1 falls from 0.84 in cross-validation to 0.63 on held-out data (Ragnarsdóttir et al.). Veterinary diagnostic coding falls from weighted F1 0.824 to 0.634 out of domain (DeepTag). A method that tells a team which finer splits their unlabeled data reliably contains, and how many, would show where fine annotation is worth paying for.

Core Claim

Our claim is that a consensus clustering whose base partitions are built across nuisance strata recovers hidden clinical subtypes better than either of the usual approaches. The first usual approach is a single clustering of pooled data. The second is a standard ensemble whose diversity comes from random seeds or bootstrap samples. Both of these let nuisance-driven clusters through. A "magnification-200× cluster" or a "one-patient cluster" is perfectly stable under re-seeding, because the confounder is a real, repeatable signal in the pooled data. It is not stable across strata. A cluster tied to one patient cannot replicate when that patient is absent from the fitting data. A cluster tied to one magnification has nothing to match at another magnification.

The consensus stability score also gives an answer to "how many subtypes does this cohort support". That question remains open in ensemble clustering. For example, choosing DivClust's target diversity without labels is unsolved (DivClust).

Relation to Prior Work

Two papers motivate the idea by showing that consensus clustering can find clinically meaningful subtypes. One builds an ensemble over resting-state connectivity features to subtype depression across centers (depression ensemble hybrid framework). The other aggregates partitions from random gene-expression projections to find cancer classes (projective clustering ensemble). Neither uses coarse supervision as scaffolding, and neither uses nuisance-stratified diversity. Their diversity is algorithmic or comes from random projections, so they cannot tell confounded clusters from biological ones. We differ on exactly those two points.

Three papers supply the machinery. The first is the co-occurrence matrix, which smooths initialization sensitivity (Yukinawa et al.). The second is the median-partition view of consensus (Clustering aggregation). The third is Hungarian matching of cluster prototypes from the Wasserstein-barycenter formulation (Qin et al.). We reuse that matching to align clusters fit on disjoint strata. When cohorts are large, the bipartite formulation of Huang et al. removes the quadratic memory cost.

The supervised fine-grained models constrain the target. CSDCNN shows that metric structure matters for the eight BreaKHis subtypes (Han et al.). LAT shows that adjacent diabetic-retinopathy grades hinge on subtle, non-dominant lesions (Lesion-Aware Transformers). This warns us that clusters formed on the most salient feature may miss low-contrast distinctions. We treat both as label-hungry upper bounds, not as competitors in the same setting.

Research Program

The pipeline has three stages. First, train an encoder with only the coarse label, for example benign/malignant on BreaKHis or referable/non-referable on EyePACS. We also include a frozen generic encoder as an alternative. Second, inside each coarse class, partition the data into nuisance strata $D_1,\dots,D_M$. On BreaKHis the strata are magnification crossed with patient-disjoint folds. Across EyePACS and Messidor-1 the stratum is the dataset of origin. Fit a soft clustering on each stratum separately, and extend it to every other point by nearest-prototype assignment.

Third, build a cross-stratum co-occurrence that counts agreement only on points outside the strata the two partitions were fit on:

$$ S_{ij} = \frac{1}{|\mathcal{P}_{ij}|}\sum_{(m,m')\in \mathcal{P}_{ij}} \mathbb{I}\big(\pi_m(x_i)=\pi_m(x_j)\big)\,\mathbb{I}\big(\pi_{m'}(x_i)=\pi_{m'}(x_j)\big) $$

Here $\pi_m$ is the partition fit on stratum $D_m$, $\mathcal{P}_{ij}$ is the set of stratum pairs for which neither $x_i$ nor $x_j$ was in the fitting data, and $\mathbb{I}$ is the indicator function. $S_{ij}$ is therefore high only when two independently fit, nuisance-disjoint clusterings both group $i$ with $j$ on unseen data. We meta-cluster $S$ spectrally. The number of sub-strata is chosen as the $k$ that maximizes mean within-cluster $S$ minus a null value obtained by permuting stratum labels. Clusters whose matched prototypes appear in fewer than a set fraction of strata are reported as non-replicating and are not called subtypes.

The analysis has three parts. We report how well the recovered clusters match hidden fine labels. We report how strongly they still correlate with the nuisance variable. And we test whether the selected $k$ matches the clinical taxonomy, which is four subtypes per coarse class on BreaKHis and the grade structure on EyePACS. For ordinal targets we additionally check whether consensus clusters can be ordered along the coarse classifier's score so that they agree monotonically with true grades.

Baselines

The naive reference point is k-means run once on pooled within-class embeddings from the same encoder, with $k$ set to the true number of fine classes. Giving it the true $k$ is generous to this baseline.

The strongest unsupervised competitor is a multi-head deep clustering ensemble in the style of DivClust, trained on pooled within-class data and combined by co-occurrence consensus. It is the strongest comparison because it produces controlled-diversity ensembles cheaply, at 10–50% overhead versus 2000% for independent models. It reports CIFAR-10 clustering accuracy rising from 79.0% to 81.9%. No reported result was found on Lacuna for DivClust or any consensus method on BreaKHis or EyePACS subtype recovery. We will run DivClust ourselves with the same encoder initialization. We sweep its target diversity over the same grid and with the same GPU-hour budget as our method, and report its best setting chosen by our unsupervised stability criterion, not by labels.

The ablation that isolates our key ingredient is an identical pipeline in which strata are random subsets of equal size instead of nuisance-defined subsets. Everything else is held fixed: encoder, base clusterer, number of partitions, and meta-clustering. Beating this ablation is the result the proposal stands on.

The supervised upper bound is CSDCNN's reported 93.2% patient-level and 93.8% image-level eight-class accuracy on BreaKHis, with an AlexNet softmax at about 83% (Han et al.). For EyePACS grading the upper bound is LAT's quadratic weighted kappa of 0.884 (Lesion-Aware Transformers). To compare against these, we train a linear probe on our consensus pseudo-labels after Hungarian mapping to true classes using only a small labeled calibration set, and report how much of the gap to these numbers it closes.

Beating the unsupervised baselines means higher patient-level adjusted Rand index against hidden subtypes. It also means lower normalized mutual information with magnification, patient, or dataset, both under patient-disjoint evaluation.

Why not just…?

Why not just stain-normalize and run k-means on a strong pretrained encoder? This is the most serious threat. If normalization plus a good encoder already removes the confounders, stratified consensus adds little. We include this pipeline as a baseline and as an input variant to our method. Normalization addresses color but not magnification or patient identity. On BreaKHis, where each patient carries a single subtype, a patient-identity cluster can look like a subtype. If normalized k-means matches our ARI and our low nuisance NMI, we will report that the ensemble is unnecessary for this dataset.

Why not just ensemble over random seeds or bootstraps? The ensemble literature motivates seed diversity as a way to escape local minima (Yukinawa et al.). Confounded clusters, however, are global structure, not local-minimum noise, so re-seeding should reproduce them faithfully. The random-strata ablation tests this directly. If it matches the nuisance-stratified version, our core claim is false.

Why not just collect fine labels and train a supervised model? Where labels exist, one should, and CSDCNN's 93.2% sets that bar. The proposal targets the earlier decision of whether a coarse archive contains replicable finer structure worth labeling. It also targets the cross-site setting, where supervised fine-grained models drop sharply (Ragnarsdóttir et al.). We measure how many calibration labels the pseudo-label probe needs to reach a given fraction of the supervised upper bound.

Why not just use a mixture model with an information criterion to choose $k$? Such criteria score fit on pooled data, so they reward splitting along strong nuisance axes. We run a Gaussian mixture with BIC on the same embeddings and check whether its selected $k$ tracks the clinical taxonomy or the nuisance variable's cardinality.

Proposed Experiments

On BreaKHis, the coarse label is benign/malignant and the hidden labels are the eight subtypes. We use five patient-disjoint folds crossed with four magnifications as strata. Metrics are patient-level ARI and NMI against subtypes, NMI against magnification and patient, and the selected $k$ per coarse class. On EyePACS with Messidor-1, the coarse label is referable versus non-referable and the hidden labels are grades. Dataset of origin and image-quality bins form the strata. Metrics add Kendall correlation between the ordered clusters and grade. The Kurume and Nagoya lymphoma cohorts (Hashimoto et al.) would provide a natural two-site test if access is obtained, and we treat them as optional.

There are two decisive checks. The first is that nuisance-stratified consensus beats the random-strata ablation on subtype ARI while showing clearly lower nuisance NMI on BreaKHis. The second is that its stability-selected $k$ lands on the clinical count more often than BIC or DivClust's stability curve across folds. If both fail, the idea that cross-stratum replication separates disease from confounder does not hold in these data.