Fairness in Recommender Systems: Research Landscape and Future Directions

Yashar Deldjoo, Dietmar Jannach, Alejandro Bellogin, Alessandro Difonzo, Dario ZanzonelliarXiv 2022 · DOI · arXiv

TL;DR

This survey analyzes over 150 recent papers to map the research landscape of fairness in recommender systems (RS). It reveals a field dominated by technical, algorithm-centric approaches that often optimize abstract metrics while neglecting the complex social, normative, and multi-stakeholder realities of what "fairness" actually means in practice.

What Problem Does This Paper Address?

Recommender systems are no longer just "benevolent" tools helping us find movies; they are powerful gatekeepers that influence our beliefs, economic opportunities, and social interactions. However, these systems can inherit or amplify biases from data, leading to unfair outcomes for different groups (e.g., gender bias in job recommendations or popularity bias that buries niche creators).

The authors identify a significant gap: while the volume of research is exploding, most of it is "stuck in an abstraction trap." Researchers often treat fairness as a purely mathematical optimization problem, ignoring the underlying ethical claims and the subjective perceptions of the humans actually using these systems. This survey aims to quantify these trends and point the way toward more impactful, interdisciplinary research.

Methods

As a survey paper, the authors employed a semi-systematic methodology to identify and categorize 157 relevant peer-reviewed publications. They developed a multi-dimensional taxonomy to evaluate how the community operationalizes fairness:

  • Level of Fairness: Group (comparing averages across segments) vs. Individual (treating similar people similarly).
  • Stakeholder Perspective: Single-sided (usually just the consumer) vs. Multi-sided (considering consumers, providers, and the platform).
  • Time Horizon: Static (one-shot evaluation) vs. Dynamic (considering feedback loops over time).
  • Causal Reasoning: Associative (correlation-based) vs. Causal (investigating "why" through counterfactuals).
  • Implementation Phase: Pre-processing (data debiasing), In-processing (algorithmic constraints), or Post-processing (re-ranking results).

Datasets

The survey highlights a major bottleneck in the field: dataset availability. * MovieLens: Strikingly, more than one-third of the analyzed papers rely on MovieLens datasets. The authors note this is problematic because these datasets often lack the sensitive demographic attributes (like race or religion) necessary for real-world fairness auditing. * Domain Concentration: Research is heavily skewed toward Video/Music (53 papers) and E-commerce (11 papers). * Synthetic Data: Due to the lack of real-world data with sensitive attributes, many researchers resort to synthetic data or artificially injecting "protected groups" into existing datasets, which may not reflect real-world social dynamics.

Key Insights and Ideas

  • The Abstraction Trap: Most technical papers assume a "gold standard" target distribution is given. They focus on how to reach that target (e.g., re-ranking) but rarely discuss why that specific distribution is fair for that specific application.
  • Technical Dominance: 83% of the literature consists of technical papers (algorithms and metrics), while only 17% are conceptual. This suggests the field is rushing to build solutions before fully defining the problems.
  • Single-Sided Bias: Despite the rise of "multi-stakeholder" platforms like Uber or Airbnb, 87.4% of research still focuses on a single side of the market (usually the consumer), ignoring potential unfairness toward providers or creators.
  • The "Re-branding" Trend: The authors observe that some researchers are simply re-branding older concepts like "diversity" or "calibration" as "fairness" without engaging with the normative ethical claims that distinguish fairness from general system quality.

Key Results

The quantitative breakdown of the current landscape reveals a lack of diversity in research methodology:

  • Static vs. Dynamic: 84% of studies are static. They ignore the "feedback loops" where biased recommendations today lead to even more biased training data tomorrow.
  • Associative vs. Causal: A staggering 97% of papers use associative methods. Only 3% attempt to use causal inference to understand the root causes of unfairness.
  • Evaluation Gap: 70% of technical papers rely solely on offline experiments. User studies (6 papers) and field tests (3 papers) are vanishingly rare, meaning we have very little data on whether "mathematically fair" recommendations are actually perceived as fair by users.
  • Reproducibility: The majority of papers do not share their code or data, hindering the progress of the field.

Limitations and Future Work

The authors identify four primary challenges for the next generation of RS research:

  1. Realistic Definitions: Moving beyond mathematical parity to definitions that account for human perception and domain-specific needs.
  2. Better Data: Developing datasets that include sensitive attributes or using advanced simulation/data augmentation to mimic realistic scenarios.
  3. Reciprocal Settings: Exploring fairness in "people-to-people" systems (like dating or job matching) where both sides must "opt-in" to a match.
  4. Fairness Auditing: Developing robust ways to audit black-box commercial systems, despite the technical and legal hurdles (e.g., platforms banning auditing bots).

Ultimately, the paper calls for a multidisciplinary shift, urging computer scientists to collaborate with sociologists, psychologists, and legal experts to ensure that "fairness" in AI reflects actual human values rather than just optimized loss functions.

Summary updated

Abstract

Recommender systems can strongly influence which information we see online, e.g., on social media, and thus impact our beliefs, decisions, and actions. At the same time, these systems can create substantial business value for different stakeholders. Given the growing potential impact of such AI-based systems on individuals, organizations, and society, questions of fairness have gained increased attention in recent years. However, research on fairness in recommender systems is still a developing area. In this survey, we first review the fundamental concepts and notions of fairness that were put forward in the area in the recent past. Afterward, through a review of more than 160 scholarly publications, we present an overview of how research in this field is currently operationalized, e.g., in terms of general research methodology, fairness measures, and algorithmic approaches. Overall, our analysis of recent works points to certain research gaps. In particular, we find that in many research works in computer science, very abstract problem operationalizations are prevalent and questions of the underlying normative claims and what represents a fair recommendation in the context of a given application are often not discussed in depth. These observations call for more interdisciplinary research to address fairness in recommendation in a more comprehensive and impactful manner.
BibTeX
@misc{deldjoo2022fairness,
  author = {Yashar Deldjoo and Dietmar Jannach and Alejandro Bellogin and Alessandro Difonzo and Dario Zanzonelli},
  title = {{Fairness in Recommender Systems: Research Landscape and Future Directions}},
  year = {2022},
  eprint = {2205.11127},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2205.11127},
}

Loading related content...

Figure