Abstract
FairFuse is an interactive visualization system designed to help decision-makers merge multiple, potentially biased rankings into a single "fair" consensus. By integrating algorithmic fairness metrics with a human-in-the-loop interface, it allows users to balance stakeholder preferences against social equity goals in high-stakes scenarios like hiring or scholarship distribution.
Introduction
Ranking is a fundamental component of decision-making, from selecting job candidates to awarding scholarships. Often, multiple stakeholders provide individual rankings that must be aggregated into a single consensus. This process faces two primary challenges: divergent preferences among rankers and systemic biases against protected groups (e.g., based on race or gender). While fair rank-aggregation algorithms exist, they often function as "black boxes," creating a gap between complex mathematical models and the practical needs of decision-makers who must audit and refine these results.
The FairFuse System
FairFuse introduces a visual analytics workflow that operationalizes the Fair-Copeland Algorithm. The system is designed around three main pillars: fairness metrics, interactive visualization, and human-in-the-loop editing.
Fairness Metrics and Visualization
The system utilizes two key metrics to quantify equity:
- Favored Pair Representation (FPR): Measures how fairly a specific group is treated by identifying over- or under-advantaged clusters.
- Attribute Rank Parity (ARP): Quantifies the overall statistical parity across all groups.
These metrics are supported by several interactive components:
- Ranking Exploration View: Employs a parallel-coordinates style to track candidate movement between different rankings.
- Group Fairness View: Combines dot plots and heatmaps to visualize which groups are being "pushed down" or "pulled up" in the ranking.
- Similarity Matrix: Uses Kendall-Tau distance to show the deviation of the consensus from original stakeholder inputs.
Key Insights and Ideas
A core strength of FairFuse is its ability to visualize the trade-off between Agreement (alignment with rankers) and Fairness (statistical parity). As users adjust the "Fairness Threshold," they can observe how the consensus shifts away from biased inputs.
Furthermore, the system explicitly identifies "over-advantaged" groups—those appearing at the top of a ranking more frequently than statistical parity would predict. This helps decision-makers spot systemic favoritism that might otherwise be overlooked. In cases involving small sample sizes (e.g., a protected group with only one member), the tool allows for manual rank adjustments, ensuring that individuals are not unfairly penalized by purely algorithmic models.
Evaluation and Results
The authors demonstrated FairFuse through two primary datasets: a Scholarship Distribution scenario involving 60 students and an Employee Bonus Distribution scenario using IBM HR data.
Qualitative evaluations showed that the system effectively helped users identify biased rankers and iteratively refine consensus rankings. By using the fairness slider, users could significantly reduce the ARP score while maintaining high agreement with stakeholders. The "Compressed Ranking View" also proved effective in reducing visual clutter for larger lists, preventing the "hairball" effect common in parallel coordinate plots.
Limitations and Future Work
Current limitations include a focus on single protected attributes, whereas real-world fairness is often intersectional. The authors also noted that the similarity matrix may face scalability issues with a high number of stakeholders. Future work will explore intersectional fairness, support for multiple ranking algorithms, and the ethical implications of "fair-washing," where metrics might be used to mask underlying biases.
Summary updated