Finding Rare Subgroups by Their Place in Hyperbolic Space

Image and text classifiers often learn a shortcut: a feature that predicts the label in most training data but is not the actual cause. Water backgrounds predict "waterbird," and the word "not" predicts "contradiction." The standard fix trains the model to do well on its worst-performing subgroup. That fix needs every training example tagged with its spurious attribute, and such tags are expensive, subjective, and sometimes unavailable. Label-free methods try to infer the subgroups instead. They look at an auxiliary model's mistakes or at how examples cluster in its feature space, and they upweight the rare examples that contradict the shortcut. These methods all work in ordinary flat feature spaces. The literature file notes that they degrade most when the contradicting examples are extremely rare, around half a percent of a class or fewer.

We propose to discover these subgroups inside hyperbolic space instead. Hyperbolic space is a curved geometry whose volume grows exponentially with distance from the center, so it can hold a tree of many small branches without squeezing them together. The data have a natural two-level shape: each class splits into a large mode where the shortcut holds and a few small modes where it fails. Our bet is that hyperbolic space keeps those small modes as separate, findable clusters in the regime where flat-space methods merge them into the majority. We will also test whether this bet is wrong, as it would be if hyperbolic distance from the center turns out to be nothing more than a relabeled confidence score.

Core Claim

We train a deliberately shortcut-prone auxiliary model whose final features live on the Lorentz model of hyperbolic space. We then cluster each class's examples by hyperbolic geodesic distance, meaning distance measured along the curved space. Clusters that sit near the origin and are small get labeled as inferred minority groups. Running worst-group training on these inferred groups should match label-free Euclidean methods at normal minority rates. It should beat them by a growing margin as the minority fraction shrinks toward 0.1%.

The mechanism has a specific prediction. The advantage should come from hyperbolic clustering separating several small modes: which background, which subtype. It should not come from hyperbolic radius ranking examples by difficulty. A softmax margin already does that ranking. If only the radius helps, the hyperbolic part is decoration and we will say so.

Relation to Prior Work

The label-free debiasing line motivates this work, and it also supplies our strongest competitors. Learning from Failure trains a bias-amplified network with generalized cross-entropy and upweights the examples it gets wrong. It reweights with a single scalar per example and never looks at feature geometry. Let Samples Speak (NSF) observes that shortcut-aligned examples cluster tightly around class centroids while contradicting examples drift toward the decision boundary, and it removes spurious directions with a linear projection. Class-Conditional Distribution Balancing reweights examples under a single-Gaussian assumption per class. Its successor, FG-CCDB with multi-stage retraining, breaks each class into discrete modes using the confusion matrix of an overfitted proxy model. The file records that this successor struggles when minority modes are extremely sparse.

uLA builds a biased linear probe on self-supervised features, adjusts logits, and introduces a proxy balanced-accuracy criterion. That criterion lets hyperparameters be chosen without group labels, and we adopt it for every method so the comparison is honest. All five methods share one assumption we relax: that minority structure is recoverable from Euclidean distances, Gaussian fits, or confusion counts. None of them asks whether a geometry built for trees would keep rare modes apart.

The hyperbolic line supplies our method. Poincaré Embeddings established that hyperbolic space embeds hierarchies in low dimensions with little distortion. PE-HCM is the closest template. It clusters coarse-class features into pseudo-subcategories in the Poincaré ball and trains with margins at the instance, fine, and coarse levels. It does this to recover fine-grained categories such as dog breeds, not spurious subgroups, and it never measures worst-group accuracy. We reuse its idea of hallucinated subclasses, but aim it at shortcut-defined groups. Robust Hyperbolic Learning with Curvature-Aware Optimization supplies the Riemannian AdamW optimizer and the distance-rescaling tricks that keep training free of numerical failures; its "robust" means numerically stable, not group-robust. HypEE uses distance from the origin as a confidence signal and uses entailment cones, which are nested cone-shaped regions encoding "A includes B," between early and late network exits. Its "shortcut" refers to exiting the network early to save compute. It therefore motivates our radius readout but is also the reason we must show that radius alone is not the whole story.

Two results constrain the design. Group DRO shows that overparameterized networks reach zero loss on every group, so the auxiliary model must be early-stopped or strongly regularized before its geometry is read. The file also notes that failure mining produces nothing when the shortcut is 100% complete. Our method inherits this limit, because a mode with no members cannot be clustered, and we do not claim otherwise.

Research Program

The auxiliary model is a pretrained ResNet-50, or BERT for text, with a small projection head onto a low-dimensional Lorentz hyperboloid. We train it with class labels only, using generalized cross-entropy to amplify the shortcut, as in LfF, and a PE-HCM-style hierarchical margin over pseudo-subclasses that are re-estimated every few epochs. We stop training early, choosing the stopping point with uLA's proxy balanced accuracy.

Within each class, we then run hyperbolic k-means with Fréchet-mean centroids. A Fréchet mean is the point minimizing summed squared geodesic distance to the cluster's members. Each cluster gets two scores: its mean radius and its size. Clusters that are both small and near the origin become inferred minority groups; everything else is the majority group. The number of clusters and the radius threshold are chosen by the same proxy criterion. A second-stage model is then trained with Group DRO over the inferred groups, using the strong weight decay that the Group DRO paper found necessary.

The analysis has three parts. First, we treat the inferred groups as a detector and measure how well they recover true minority membership, using AUROC and cluster purity. True group labels are used here and nowhere else in training or selection. Second, we check whether separate small clusters line up with distinct true subgroups, which is the multi-mode claim. Third, we test whether a spurious attribute appears as a shared direction across classes. On Waterbirds this would mean the "land background" clusters of both classes point in similar tangent directions at the origin. That result would be the first evidence that the shortcut is being organized as a parent concept.

Baselines

The naive reference point is ERM, meaning plain average-loss training, with the same backbone and budget. The Group DRO paper reports its worst-group accuracy as 60.0% on Waterbirds and 41.1% on CelebA blond males.

The strongest label-free competitors are NSF, CCDB, and uLA. NSF reports 91.12% worst-group accuracy on Waterbirds, CCDB reports 90.48%, and uLA reports 86.1%. We found no reported result on Lacuna for FG-CCDB on Waterbirds, so we will run it ourselves; it is the most direct test of our sparsity claim. The supervised ceiling is Group DRO with true group labels, reported at 90.5% on Waterbirds in the Group DRO paper and at 86.7% on CelebA's hardest group.

Many of the published label-free numbers may rely on group-labeled validation, the practice the file calls "cheating in validation." We will therefore rerun NSF, CCDB, FG-CCDB, uLA, and LfF under identical conditions: the same backbone, the same number of epochs, 20 hyperparameter trials each, and selection by uLA's proxy criterion for all methods. We report these reruns alongside the published figures.

At the published minority rates, beating the baselines means matching the best rerun label-free method within noise. On the scarcity variants, it means a worst-group accuracy gain that is larger than the spread across three seeds and that grows as the minority fraction falls.

Three ablations isolate the geometry. The first is a Euclidean twin: an identical pipeline with a flat projection head, Euclidean k-means, and the same margin loss. The second is a radius-only variant, which ranks examples by hyperbolic radius and thresholds them without clustering. The third is a margin-only variant, which uses the auxiliary model's softmax margin in place of radius.

Why not just…?

Why not just use the auxiliary model's confidence or loss to find minority examples? This is the most serious threat, because HypEE already treats hyperbolic radius as a confidence score. If the radius-only variant matches the full method, and radius is almost perfectly rank-correlated with softmax margin, then hyperbolic geometry adds only a ranking that LfF already gets from loss. The clustering claim would fail in that case. We will report that result and drop the geometric claim rather than defend it.

Why not just cluster in Euclidean space, as NSF and FG-CCDB effectively do? At normal minority rates we expect this to work about as well, and NSF's 91.12% suggests there is little room left. The case for hyperbolic space rests entirely on the scarcity regime, where exponential volume should keep tiny modes from being pulled into the majority. The Euclidean-twin ablation, run on Waterbirds with minority groups subsampled to 1%, 0.5%, and 0.1%, settles this question directly.

Why not just train a hyperbolic classifier with ERM and skip group inference? Curved geometry alone might reduce reliance on the shortcut. We run a hyperbolic-head ERM model as an additional baseline. If it closes most of the gap on its own, the useful ingredient is the geometry as an inductive bias rather than subgroup discovery, and the proposal would be reframed around that finding.

Proposed Experiments

We use Waterbirds and CelebA, plus MultiNLI and CivilComments for text, with worst-group accuracy and average accuracy as the primary metrics. The key experiment is the controlled scarcity sweep on Waterbirds and CelebA. We subsample the minority groups in the training set while leaving test sets untouched, then plot worst-group accuracy and minority-detection AUROC against minority fraction for the full method, the Euclidean twin, the radius-only variant, FG-CCDB, and NSF.

Three outcomes are decisive. First, the full method's detection AUROC should fall more slowly than the Euclidean twin's as scarcity increases. Second, the full method should beat the radius-only variant, which would show that the gain comes from clustering rather than confidence. Third, inferred clusters on Waterbirds should align with distinct true subgroups at purity well above what the Euclidean twin achieves at the same number of clusters. If the first check fails, the proposal fails. If the first holds but the second fails, the honest conclusion is that hyperbolic radius is a well-behaved confidence score, not a subgroup detector. If all three hold, we will have a group-label-free method whose advantage appears where current methods break down.