From Bias to Balance: Fairness-Aware Paper Recommendation for Equitable Peer Review

Uttamasha Oyshi, Susan Gauch

TL;DR

Researchers have developed Fair-PaperRec, a neural network-based re-ranking system that uses a differentiable fairness loss to mitigate demographic bias in academic peer review. By specifically targeting underrepresented groups based on race and country, the system increased diverse participation by up to 42% while maintaining—and in some cases even enhancing—the overall scholarly quality of accepted papers.

What Problem Does This Paper Address?

Even with double-blind review, the academic "meritocracy" is often tilted. Systemic biases regarding an author’s race, nationality, or institutional prestige frequently leak through via writing styles or citation patterns. This results in a "rich-get-richer" cycle where researchers from dominant regions and demographics are consistently over-selected, while high-quality work from underrepresented groups is overlooked.

While recommendation algorithms are increasingly used to manage the flood of conference submissions, most are optimized solely for predictive accuracy. If the training data is biased, the algorithm simply automates and accelerates that bias. This paper tackles the challenge of building a "post-review" recommender that explicitly balances scholarly utility with demographic justice.

Methods

The authors introduce Fair-PaperRec, a framework built on a lightweight Multi-Layer Perceptron (MLP). The core innovation is the integration of a differentiable fairness loss into the standard prediction objective.

  • The Objective Function: The model minimizes a combined loss: $L_{total} = L_{prediction} + \lambda \cdot L_{fairness}$.
  • The Fairness Regularizer ($\lambda$): This hyperparameter acts as a "dial" that conference organizers can turn to balance the trade-off between historical accuracy and demographic parity.
  • Intersectional Approach: Unlike many systems that handle one attribute at a time, Fair-PaperRec can manage multiple protected attributes (like race and country) simultaneously using weighted combined losses.
  • Two-Stage Validation: The researchers first tested their hypothesis on synthetic data to find the "sweet spot" for $\lambda$ under different bias regimes (fair, moderate, high), then applied those lessons to real-world conference data.

Datasets

The study utilized two distinct types of data:

  1. Synthetic Datasets: Custom-built datasets with tunable distributions to simulate varying levels of demographic skew.
  2. Real-World Conference Data: Submissions and metadata from three major Human-Computer Interaction (HCI) venues: SIGCHI 2017, DIS 2017, and IUI 2017. These datasets included paper-level features and author-level metrics like the h-index, which served as a proxy for scholarly "utility."

Key Insights and Ideas

  • Fairness as a Quality Booster: Perhaps the most surprising finding is that in highly biased environments, fairness regularization doesn't just promote equity—it acts as a quality regularizer. By forcing the model to look past demographic "noise," it uncovers high-quality, high-impact work that was previously under-selected.
  • The "Sweet Spot" is Regime-Dependent: There isn't a one-size-fits-all setting for fairness. In high-bias scenarios, a stronger $\lambda$ (around 3) is needed to reach a peak where both fairness and utility improve. In already "fair" systems, too much regularization can lead to over-correction.
  • Macro vs. Micro Dynamics: The system is highly effective at improving Macro Gain (overall group representation) but shows more volatility in Micro Gain (individual author outcomes). This suggests that while we can balance the "portfolio" of a conference, ensuring equity for every individual remains a complex challenge.
  • Attribute Sensitivity: Race and country respond differently to regularization. In the datasets studied, race required more aggressive intervention to achieve balance compared to geographic origin.

Key Results

  • Participation Spike: In real-world conference scenarios, Fair-PaperRec achieved up to a 42.03% increase in participation from underrepresented groups.
  • Stable Utility: This massive boost in diversity came at a negligible cost to quality; the overall utility (measured by weighted h-index) changed by at most 3.16% compared to historical selections.
  • Win-Win Scenarios: In synthetic "high-bias" tests, the model actually produced a 15.32% gain in utility while simultaneously improving diversity, proving that equity and excellence are not mutually exclusive.

Limitations and Future Work

  • Proxy Metrics: The authors acknowledge that the h-index is an imperfect proxy for quality, as it is itself influenced by seniority and field-specific biases.
  • Gender Exclusion: In this specific study, gender was excluded from the fairness constraints because the initial distribution in the chosen datasets was relatively balanced. Future iterations aim to make the model fully "gender-aware."
  • Causal Pathways: The current MLP model treats attributes as features but doesn't explicitly model the causal reasons why certain groups are disadvantaged. The authors suggest that future work involving graph-based architectures or causal inference could provide deeper insights into the mechanics of peer-review bias.
  • Human-in-the-loop: Moving from a research prototype to a real-world tool will require transparent reporting and governance mechanisms to ensure the system remains accountable to the community it serves.

Summary updated

Abstract

Despite frequent double-blind review, systemic biases related to author demographics still disadvantage underrepresented groups. We start from a simple hypothesis: if a post-review recommender is trained with an explicit fairness regularizer, it should increase inclusion without degrading quality. To test this, we introduce Fair-PaperRec, a Multi-Layer Perceptron (MLP) with a differentiable fairness loss over intersectional attributes (e.g., race, country) that re-ranks papers after double-blind review. We first probe the hypothesis on synthetic datasets spanning high, moderate, and near-fair biases. Across multiple randomized runs, these controlled studies map where increasing the fairness weight strengthens macro/micro diversity while keeping utility approximately stable, demonstrating robustness and adaptability under varying disparity levels. We then carry the hypothesis into the original setting, conference data from ACM Special Interest Group on Computer-Human Interaction (SIGCHI), Designing Interactive Systems (DIS), and Intelligent User Interfaces (IUI). In this real-world scenario, an appropriately tuned configuration of Fair-PaperRec achieves up to a 42.03% increase in underrepresented-group participation with at most a 3.16% change in overall utility relative to the historical selection. Taken together, the synthetic-to-original progression shows that fairness regularization can act as both an equity mechanism and a mild quality regularizer, especially in highly biased regimes. By first analyzing the behavior of the fairness parameters under controlled conditions and then validating them on real submissions, Fair-PaperRec offers a practical, equity-focused framework for post-review paper selection that preserves, and in some settings can even enhance, measured scholarly quality.
BibTeX
@misc{oyshi2026from,
  author = {Uttamasha Oyshi and Susan Gauch},
  title = {{From Bias to Balance: Fairness-Aware Paper Recommendation for Equitable Peer Review}},
  year = {2026},
  eprint = {2602.22438},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2602.22438},
}
Figure