Hyperbolic Radius as a Label-Free Guide for Fair Pruning

When a neural network is compressed by deleting most of its weights (pruning) or storing them in fewer bits (quantization), overall accuracy often barely moves while accuracy on rare classes and minority demographic groups falls sharply. The methods that fix this all need to know, for every training image, which group it belongs to. That information is often missing, noisy, or legally restricted. This note proposes getting a substitute signal from geometry. Hyperbolic embeddings place data in a space whose volume grows exponentially outward from the center. In such embeddings, generic, prototypical examples sit near the center and unusual, specific ones drift toward the edge. We ask whether distance from the center (the "radius") in a hyperbolic embedding trained without group labels can tell a compression algorithm which examples it is about to fail. If it can, we can protect those examples during pruning as though we had group labels.

The claim is testable, and it could easily be wrong. Radius might track nothing more than classifier confidence, or plain density in an ordinary Euclidean embedding. So the program is built around one decisive comparison. Before any compression happens, we score each sample by hyperbolic radius and by the strongest Euclidean rarity scores. We then measure which score better predicts which samples, and which demographic groups, lose accuracy after compression. We then plug the winning score into an existing fairness-aware pruner in place of true group labels.

Core Claim

Consider a hyperbolic embedding learned with a label-free, self-supervised objective on frozen backbone features. Its radial coordinate ranks examples by how atypical they are relative to the structure of the data. That ranking predicts per-example accuracy loss under pruning better than Euclidean distance-to-neighbors or dense-model confidence. Grouping examples into radius bands ("shells") and preserving each shell's share of weight importance should recover much of the worst-group benefit of label-supervised fair pruning, with no demographic labels at all.

The claim has two parts. The first is a diagnostic: does radius predict compression damage? The second is a method: does shell-based protection close the fairness gap? The second part is only worth pursuing if the first part succeeds.

Why this intersection, and what is already known

Several hyperbolic papers motivate the radius-as-specificity idea. Poincaré Embeddings for Learning Hierarchical Representations embeds taxonomies in the unit ball and shows general concepts settle near the origin and leaves near the boundary. APo-VAE reports the same radial ordering in a learned text latent space. Neither paper connects radius to which examples a compressed model will fail.

The closest work runs in the opposite direction from ours. Hyperbolic Dataset Distillation maps features into the Lorentz model of hyperbolic space, a numerically stable representation of the same geometry. It matches centroids there and finds that the outer 80% of samples by radius can be thrown away with little loss in aggregate distillation accuracy. That result treats peripheral examples as expendable noise. Pruning-fairness work says the opposite: peripheral examples are exactly the ones that matter. FairGRAPE preserves each demographic group's share of gradient-based weight importance in every layer. Balancing Act bounds each group's excess accuracy drop through constrained optimization. LTAP shields weights that tail classes depend on. All three need group or class labels, and none offers a geometric account of where vulnerable examples live. We use the radius to identify examples to protect rather than to discard, and we first audit whether the distillation paper's discarded 80% is demographically skewed.

Two papers constrain the design. Clipped Hyperbolic Classifiers shows that features pushed toward the boundary of the Poincaré ball (the bounded unit-ball version of hyperbolic space) lose their gradients. Its fix caps feature norms before mapping them into hyperbolic space. That cap may squash exactly the radial spread we want to read, so the clipping radius becomes an experimental variable rather than a fixed default.

Identifying and Mitigating Gender Bias in Hyperbolic Word Embeddings cuts against us. It finds that hyperbolic word embeddings inherit Euclidean gender bias almost exactly, with a correlation of 0.952. Hyperbolic geometry is therefore not a fairness cure in itself. Our proposal does not rely on it being one; it uses geometry only as a detector.

Hyperbolic Aware Minimization shares the word "hyperbolic" and the word "bias," but it concerns an optimizer's pull toward sparse weights. It reports aggregate accuracy only, so it is a compatible pruning backbone rather than a competitor.

Research Program

We start with a pretrained vision backbone, such as ResNet-34 as in FairGRAPE. On its frozen penultimate features we train a small Lorentz-model projection head with a contrastive objective over image augmentations. This head sees neither target labels nor demographic labels. A deliberate consequence is that radius measures how atypical an image is within the data, not how hard it is for the task classifier. That separation matters because a gender classifier can be confident on a minority racial group and still lose accuracy there after pruning.

Each training sample $x$ receives a radius $\rho(x)$, its geodesic distance from the embedding's centroid. Samples are binned into $K$ radius quantile shells $S_1,\dots,S_K$. We then run FairGRAPE's procedure with shells substituted for demographic groups. The pruner keeps each shell's share of total importance fixed in every layer:

$$\frac{\sum_{w \in \text{kept}} I_{k,w}}{\sum_{w} I_{k,w}} \approx \text{const across } k,$$

where $I_{k,w}$ is the first-order Taylor importance of weight $w$ computed on shell $S_k$. The same shells can replace demographic groups in FairQuant's fairness loss, which assigns more bits to layers that minority groups depend on; this is a secondary quantization test.

The program has three stages. Stage one is an audit. We report the demographic composition of each radius shell on FairFace and UTKFace, and we measure how much the outer 80% discarded under the Hyperbolic Dataset Distillation rule over-represents minority groups.

Stage two is the predictive test. Before compressing, we score every test sample by each candidate vulnerability score. After magnitude pruning at 90% and 99% sparsity, we compute the area under the ROC curve (AUROC) for predicting which samples flip from correct to incorrect. We also compute the rank correlation between each group's mean score and that group's excess accuracy drop.

Stage three is the method. We run shell-guided pruning and compare worst-group accuracy against the baselines below. Throughout, we sweep the clipping radius to test whether clipping destroys the radial signal.

Baselines

The naive reference point is global magnitude pruning followed by standard fine-tuning, with no fairness mechanism. Balancing Act reports that on UTKFace race prediction at 90% sparsity, this approach leaves some groups with a "10% higher accuracy drop" than average. Our method must shrink that excess drop on test data.

The strongest existing method is FairGRAPE run with true demographic labels. On FairFace with ResNet-34 at 99% sparsity it reports 91.1% overall accuracy with a gender accuracy standard deviation of 0.05 (FairGRAPE). We treat it as an oracle upper reference, not a target to beat. We count success as recovering at least half of the gap between naive pruning and oracle FairGRAPE in worst-group accuracy, using zero demographic labels. CEAG with labels (Balancing Act) is a second oracle. It reports near-zero excess gap on training data but a re-emerging gap at test time, so we compare against it on test data only.

The label-free competitors are the real contest. The first is FairGRAPE with pseudo-groups from K-means on the backbone's Euclidean features. The second is FairGRAPE with shells defined by Euclidean k-nearest-neighbor distance in an otherwise identical contrastively trained Euclidean head. This is the key ablation, since it changes only the geometry. The third uses shells defined by the dense model's per-sample loss. The fourth uses shells defined by hyperbolic radius from a head trained on target labels, which tests whether the label-free objective matters.

Random shells control for the benefit of simply spreading importance across arbitrary partitions. For long-tailed data we compare against LTAP, which reports 34.1% tail accuracy on CIFAR-100-LT (imbalance ratio 50) versus 23.8% for its RReg baseline (LTAP). For quantization, the reference is FairQuant on Fitzpatrick17k with ResNet18. There, uniform 4-bit quantization drops average accuracy from 50.6% to 23.4%, and FairQuant at about 4 bits recovers 45.33% (FairQuant).

All methods share the same backbone, sparsity schedule, fine-tuning epochs, and calibration data. Every method, including ours, receives an equal budget of hyperparameter trials for its number of groups or shells $K$. The scoring heads for all label-free variants get identical training compute.

Why not just…?

Why not just use the dense model's loss or confidence as the vulnerability score? It is the cheapest option and it will correlate with compression damage. The reason to expect radius to add something is that demographic disparity under pruning is not the same thing as task difficulty. Minority-group images can be classified confidently yet depend on weights that global magnitude pruning removes. The loss-shell baseline measures this directly in stages two and three. If loss shells match hyperbolic shells, we will report that the geometry is unnecessary.

Why not just use Euclidean density in the same self-supervised embedding? This is the most serious threat. A k-nearest-neighbor rarity score captures "atypical" without any curvature. Our hypothesis is that Euclidean crowding compresses distances among rare, fine-grained examples, which weakens density estimates exactly in the tail. Hyperbolic space's exponential outer volume keeps those examples spread apart. The matched Euclidean-head ablation settles the question. If the AUROC difference is not significant across FairFace, UTKFace, and CIFAR-100-LT, the core claim fails, and the paper becomes a negative result about geometry plus the stage-one audit.

Why not just cluster in Euclidean space and feed the clusters to FairGRAPE? K-means produces hard, roughly equal-sized groups whose boundaries need not line up with vulnerability. Radius shells are ordered, so they specifically isolate the periphery. The K-means pseudo-group baseline tests this directly under the same pruner.

Proposed Experiments and Decisive Checks

The primary benchmarks are FairFace and UTKFace for demographic pruning with a ResNet-34 at 90% and 99% sparsity. CIFAR-100-LT tests class imbalance. Fitzpatrick17k tests 4-bit quantization. We report overall accuracy, worst-group accuracy, test-set excess accuracy gap, group accuracy standard deviation, and flip-prediction AUROC. All results are averaged over three seeds with confidence intervals.

Three results are decisive. First, stage one either shows or fails to show that the Hyperbolic Dataset Distillation periphery over-represents minority groups; either outcome is publishable on its own. Second, hyperbolic radius must beat matched Euclidean kNN rarity and dense-model loss in flip-prediction AUROC. Without that, the method stage is not justified. Third, shell-guided pruning must recover a substantial fraction of oracle FairGRAPE's worst-group gain without labels, and its test-time excess gap must be no worse than label-supervised CEAG's.

The clipping-radius sweep adds a mechanistic check. If radius is doing the work, flip-prediction AUROC and worst-group gains should shrink as clipping tightens and collapses the radial spread.