Many different classifiers reach the same accuracy. Among them, some treat demographic groups very differently and some do not. Fairness research usually responds in one of two ways. It treats this as a hazard, the "fairness lottery," where changing the random seed changes subgroup performance, or it runs a combinatorial search for the single fairest model. Engineers who design optical and mechanical structures face the same situation in another form. Many shapes produce the same physical response, so their field has learned two lessons. First, if you train a network to output one answer when many answers are valid, it returns their average, and the average is often invalid. Second, a candidate should be judged by whether it actually achieves the target, not by how close it is to one reference solution. We propose to treat building a fair classifier as an inverse design problem of this kind. The input is a target point (accuracy $a^*$, group disparity $d^*$). The output is a whole distribution of model parameters that realize that point. A generator (a conditional flow, i.e. a neural sampler that learns to transport noise into samples matching a condition) is trained against the cheap "forward map" that scores a candidate model's accuracy and disparity. It also learns to abstain on targets no model can reach, so the boundary of its abstention region is an estimate of the accuracy-fairness Pareto frontier.
The payoff is not only another way to trace that frontier. Once we can sample the full set of equally good models at a chosen trade-off point, two quantities become measurable that a single-model method cannot see. One is how much individual decisions disagree within that set. The other is which models in the set keep their fairness when moved from validation to test data. Our central hypothesis is about the second. Choosing the fairest model on validation data is a winner's-curse selection. Choosing the model at the center of the sampled set, the one whose decisions agree most with its equally good peers, should carry its fairness to test data much better.
Core Claim
The map from a model's parameters to its (accuracy, disparity) pair is many-to-one. Seen this way, the Rashomon set (the set of models whose loss is within $\epsilon$ of the best) becomes the preimage of a target in performance space. This lets us import three tools from inverse design. Tandem training scores generated models in performance space rather than parameter space. Flow matching with abstention samples diverse solutions while declining infeasible targets. The averaging-trap diagnosis predicts exactly where deterministic "preference-to-model" regressors will fail. We claim that sampling this preimage at a fixed trade-off point and deploying its consensus member gives test-set disparity closer to its validation value than seed search or single-solution constrained training, at matched accuracy.
Relation to Prior Work
The closest work also borrows inverse design for fairness, but it inverts a different object. LUCID freezes a trained classifier and runs gradient descent on inputs to find a "canonical set" of individuals the model favors. LUCID-GAN replaces that gradient search with a conditional tabular GAN so that it works on black-box models. Both invert the map from inputs to predictions, and both are audits of a fixed model. We invert the map from model parameters to (accuracy, disparity), and the product is a trained model, or a family of them, rather than a diagnostic.
Fairness work on multiplicity motivates the proposal. Be Intentional About Fairness! searches Rashomon sets of discrete models with knapsack algorithms. It shows that disparity can often be removed for less than 0.5% accuracy, and that equally accurate models flip up to 12% of individual decisions. Fairness and Sparsity Within Rashomon Sets uses exact optimization over sparse scoring systems and decision trees to prove that bias can be certifiably unavoidable. Both methods are exact, but they are limited to small discrete hypothesis classes and datasets of a few thousand samples. Our method is continuous and approximate, and it applies to differentiable model heads. On the Impact of Random Seeds on the Fairness of Clinical Classifiers both motivates and constrains the design. It shows that seed changes move subgroup AUC by nearly 15% at fixed overall AUC near 0.8 on MIMIC-III. It also shows that validation fairness barely predicts test fairness, so any selection rule we propose must be judged on held-out data, with small groups treated as noisy.
Single-solution methods for tracing the frontier are our strongest competitors. Differentiable Optimization Layers (F-Layer) projects predictions onto a convex fairness constraint set inside the network. MIFPO computes the optimal trade-off through a Bayes-optimal score without training representations. fairlib supplies Distance to the Optimal (DTO) as a model-selection metric. All three return one model, or one bound, per trade-off level. None of them describes the set of models sharing that level.
Inverse design supplies the method. We reuse tandem training against a frozen forward model (Deng et al.) and flow matching with abstention (de Campos et al.). From Richmond et al. we take the evaluation principle that candidates are scored by whether they reach the target, not by distance to a reference. Bundle Networks model preimages as fiber bundles, that is, collections of local patches rather than one global latent space. They beat GANs on preimage fidelity, for example Wasserstein-1 of 0.461 versus 1.610 on a torus. Their pairwise neighborhood computation scales quadratically with sample count, so we use them only as an ablation if fibers turn out to be disconnected.
Research Program
The object we generate is the parameter vector $\theta$ of a small classification head: logistic regression or a one-hidden-layer network. The head sits on fixed features, which are raw tabular features or frozen encoder embeddings for clinical text. This keeps the forward map cheap. Evaluating a candidate means one pass over the training split to compute accuracy $A(\theta)$ and a group disparity $D(\theta)$, such as the statistical parity difference. Training uses sigmoid-smoothed versions of both, and evaluation uses the exact metrics. A conditional flow $g(z; a^*, d^*)$ maps noise $z$ to heads. It is trained with a tandem objective,
$$\mathcal{L} = \mathbb{E}_{z,(a^*,d^*)}\big[(\tilde A(g) - a^*)^2 + (\tilde D(g) - d^*)^2\big] - \beta\, H(g \mid a^*, d^*),$$
where $\tilde A$ and $\tilde D$ are the smoothed metrics and $H$ is an entropy term that rewards spread within each condition. The entropy term is estimated with the flow's exact log-density, and it prevents the generator from collapsing to one solution per target. Targets are drawn over a grid that deliberately extends beyond what is achievable. Following the abstention scheme of de Campos et al., the generator also outputs a feasibility score, and the boundary of that score is our frontier estimate.
The averaging trap appears concretely here. Swapping hidden units of a head leaves its function unchanged, and different constraint-satisfying solutions can sit far apart in parameter space. A deterministic hypernetwork trained to map $(a^*, d^*)$ to $\theta$ will therefore average across modes. We predict it will miss its targets most where the preimage is multimodal. This is a testable claim imported directly from the physical-design literature.
Downstream, for a chosen target we draw a few hundred heads and compute three fiber-level quantities. The first is the arbitrariness rate, the fraction of individuals whose decision differs across samples. The second is the consensus member, the sample whose predictions agree most with all other samples, a medoid in prediction space. The third is the fiber ensemble, a majority vote over samples. The consensus member is our proposed deployable model.
Baselines
The naive reference point is a standard unconstrained classifier trained with ordinary loss minimization, run over 50 seeds with the validation-fairest seed selected. This is the procedure Amir et al. found to generalize poorly. The same head architecture, features, and splits are used throughout. Beating it means a smaller gap between validation and test disparity, and lower test disparity, at matched test accuracy.
The strongest frontier competitor is the F-Layer, swept across constraint levels and seeds under the same tuning budget in GPU-hours as our generator plus its training-data budget. It is strongest because it guarantees constraint satisfaction and outperforms post-hoc projection. Troxell et al. report a 1.46% mean AUC improvement over post-hoc projection on SBA, 100% constraint satisfaction, and 2–5% accuracy gains on FairFace. We run it on our tabular datasets rather than quoting these numbers. Matching its frontier within noise counts as parity. Beating it requires our consensus member to show a significantly smaller validation-to-test disparity drift at the same constraint level.
For the small sparse model classes where exact answers exist, the knapsack and exact-optimization searches of Dai et al. serve as ground truth rather than competitors. On German Credit at 2% error tolerance they report about 11% disparity for a random Rashomon model versus zero for the fairest. On Adult, statistical parity differences in the 20% Rashomon set range from near 0 to over 0.10. Our sampler, restricted to linear scoring heads, should reproduce these ranges. If it covers only a narrow band, it is missing parts of the preimage.
A preference-conditioned deterministic hypernetwork, identical to our generator but without noise or the entropy term, isolates the key ingredient of modeling the distribution rather than one answer. A conditional GAN in the style of LUCID-GAN, trained on the same targets, tests whether any generative model suffices. For MOJI and BIOS, where fairlib evaluates 14 debiasing methods by DTO, no reported result was found on Lacuna, so we will run fairlib's methods ourselves under our splits.
Why not just…?
Why not just run many seeds and keep the fairest one? Amir et al. tested exactly this on MIMIC-III and found validation fairness weakly or not at all correlated with test fairness. Dai et al. show that random draws from the Rashomon set carry large disparity. Seeds sample the set blindly, across all disparity levels. Our sampler is conditioned on the target disparity, and the consensus rule replaces argmin-on-noise selection. The first experiment below measures directly whether this difference matters.
Why not just sweep the constraint level in a constrained solver or an F-Layer? This is the real threat. If the only goal were a frontier, a convex sweep is simpler and probably at least as accurate, and we expect our frontier merely to match it. The proposal therefore stands or falls on what one model per level cannot provide: an arbitrariness estimate and a selection rule that generalizes. If seeded F-Layer solutions at a level, with their own medoid taken, transfer to test data as well as our consensus member, the generator adds nothing. We will report that outcome as a negative result.
Why not just ensemble seed models? Averaging predictions is a valid classifier, unlike averaging designs, and ensembles reduce variance, so this is a serious competitor for stability. The distinction is that a seed ensemble mixes models of very different disparity, so its own disparity is uncontrolled. A fiber ensemble mixes only models at the target. We compare test disparity and accuracy of both at equal ensemble size.
Why not just use a conditional GAN, as LUCID-GAN does? Courts and Kvinge show that conditional GANs lose preimage structure, with fiberwise Wasserstein-1 of 3.666 versus 1.917 for BundleNet on Wine Quality. We include the GAN as a baseline and measure its coverage of the Rashomon disparity range against exact enumeration.
Proposed Experiments
The datasets are Adult, COMPAS, German Credit, Default of Credit Card Clients, and Heritage Health, all tabular, plus MIMIC-III phenotyping with frozen text embeddings for the seed-lottery setting. Each uses 20 random train/validation/test splits, because small-group metrics are noisy.
The decisive check is selection transfer. At three target disparity levels we compare test disparity, its absolute drift from validation, and test accuracy across five selection strategies: the validation-best seed, the validation-best F-Layer run, the consensus member, the fiber ensemble, and the seed ensemble. Significance is computed over splits. The hypothesis fails if the consensus member's drift is not significantly below both the seed and F-Layer selections on at least the small-group datasets, COMPAS and MIMIC subgroups.
Three supporting checks follow. The first is target-hit error in performance space for the stochastic generator versus the deterministic hypernetwork, as the head widens from linear to 64 hidden units. The averaging trap predicts that the hypernetwork's error grows with width, as permutation symmetry multiplies modes, while the generator's does not. The second is coverage of the exact Rashomon disparity ranges on sparse linear heads, compared against the Dai et al. figures. The third is the frontier gap: the hypervolume and DTO of the abstention boundary versus the F-Layer sweep. If fibers prove disconnected, which shows up as bimodal consensus scores, we add a BundleNet-style ablation with local patches and report whether it changes the selection results.
References
- LUCID: Exposing Algorithmic Bias through Inverse Design
- LUCID-GAN: Conditional Generative Models to Locate Unfairness
- Be Intentional About Fairness!: Fairness, Size, and Multiplicity in the Rashomon Set
- Fairness and Sparsity Within Rashomon Sets: Enumeration-Free Exploration and Characterization
- On the Impact of Random Seeds on the Fairness of Clinical Classifiers
- Differentiable Optimization Layers for Guaranteed Fairness in Deep Learning
- Efficient Fairness-Performance Pareto Front Computation
- FairLib: A Unified Framework for Assessing and Improving Fairness
- Tandem Neural Network Based Design of Acoustic Metamaterials for Low-Frequency Vibration Reduction in Automobiles
- Generative Inverse Design with Abstention via Diagonal Flow Matching
- On the evaluation of inversion mapping performance in the acoustic domain
- Bundle Networks: Fiber Bundles, Local Trivializations, and a Generative Approach to Exploring Many-to-one Maps