TL;DR
This paper tackles the challenge of designing fair decision-making systems for AI, specifically focusing on how to collectively choose a linear ranking rule that respects the preferences of diverse user groups over time. They introduce the "angular mean" as a novel rule that surprisingly achieves long-run fairness, outperforming the standard arithmetic mean, and demonstrate that while perfect batch-level fairness is impossible, it's closely approximated with larger batch sizes.
What Problem Does This Paper Address?
In AI alignment and participatory design, a key problem is how to aggregate individual preferences into a single, consistent decision-making system that will be used repeatedly. Imagine an AI system that needs to rank job applicants, research proposals, or even medical treatments. If this system uses a fixed set of criteria (a "linear ranking rule"), it's crucial that this rule doesn't systematically disadvantage certain groups of users over time. The authors frame this as "individual proportionality" (IP): each user group should have their preferences reflected in the system's decisions proportionally to their size. The standard approach of averaging user preferences (the "arithmetic mean") has been shown to be "majoritarian," meaning it can heavily favor the majority and neglect minority viewpoints. The paper asks: can we find a single, fixed linear ranking rule that guarantees fairness for all user groups, both in the long run and ideally even on a per-decision basis?
Methods
The core of the paper revolves around defining and analyzing two notions of fairness: long-run individual proportionality (LONG-IP), where fairness is guaranteed on average over many decisions, and per-batch individual proportionality (BATCH-IP), a stricter requirement for fairness within each individual decision batch.
The authors introduce and analyze the angular mean as a new aggregation mechanism. This rule is the spherical analog of the arithmetic mean, essentially finding a consensus direction on a sphere that balances voter preferences. They prove that this angular mean guarantees LONG-IP.
They also demonstrate the impossibility of achieving perfect BATCH-IP with any fixed linear rule, but show that the gap between BATCH-IP and LONG-IP shrinks rapidly as the batch size increases. This is achieved through theoretical analysis involving geometric interpretations of agreement and concentration inequalities.
Datasets
The paper evaluates their proposed methods on three real-world preference datasets:
- Moral Machine (MMACHINE): Dilemmas involving autonomous vehicle accident choices.
- Kidney Exchange (KIDNEY): Prioritizing kidney transplant candidates.
- Food Rescue (FOODRESCUE): Allocating resources for food donations.
They also create "2D variants" of these datasets by selecting specific features to induce greater voter heterogeneity, and "heterogeneity-filtered" subsets to create challenging scenarios with highly separated voter clusters.
Key Insights and Ideas
- The Angular Mean is a Game Changer for Long-Run Fairness: The most surprising finding is that a simple, geometrically motivated rule (the angular mean) can provably satisfy long-run individual proportionality, a feat the standard arithmetic mean fails at. This suggests that geometric intuition can lead to robust fairness guarantees.
- Perfect Batch-Level Fairness is Unattainable for Fixed Rules: The paper definitively shows that no single, fixed linear rule can guarantee fairness within every single decision batch. This highlights a fundamental trade-off between consistency and perfect per-batch fairness.
- Batch Size is Key to Approximating Batch-Level Fairness: Despite the impossibility of perfect BATCH-IP, the gap between long-run and batch-level fairness shrinks dramatically as the batch size grows. This means that for practical applications where decisions are batched, the angular mean gets very close to ideal fairness.
- Homogeneity Masks Fairness Issues: When voter preferences are similar, most aggregation rules perform well. The real test comes with high disagreement, where the angular mean's superiority becomes evident.
Key Results
- The angular mean is proven to satisfy long-run individual proportionality (LONG-IP = 1) for spherically symmetric item distributions.
- It is proven that exact per-batch individual proportionality (BATCH-IP = 1) is impossible for any fixed linear ranking rule.
- However, the gap between BATCH-IP and LONG-IP for the angular mean converges to zero as the batch size m increases.
- Empirical results on real-world datasets show that when voter preferences are homogeneous, the angular mean performs similarly to other rules. But in high-disagreement regimes, the angular mean significantly outperforms the arithmetic mean and geometric median in terms of long-run IP.
Limitations and Future Work
- Spherical Symmetry Assumption: The theoretical guarantee for the angular mean's long-run proportionality relies on the assumption of spherically symmetric item distributions. While experiments suggest robustness to deviations, this remains a theoretical limitation.
- Fixed Linear Rules: The paper focuses on fixed linear ranking rules. Future work could explore nonlinear rules or rules that adapt over time, which might achieve better per-batch proportionality.
- Approximation of Optimal Rules: The paper shows that the angular mean approximates the optimal rule for the Squared Kemeny objective as batch size grows. Further investigation into this connection and its implications for proportionality could be fruitful.
- Downstream Tasks: The paper focuses on ranking. Extending these fairness notions and aggregation rules to other downstream decision tasks (e.g., selection, allocation) is a natural next step.
Summary updated