Generative Discrimination: What Happens When Generative AI Exhibits Bias, and What Can Be Done About It

Philipp Hacker, Brent D. Mittelstadt, Frederik J. Zuiderveen Borgesius, Sandra WachterarXiv 2024 · DOI · arXiv

TL;DR

Generative AI (genAI) systems can perpetuate and even amplify societal biases, leading to discriminatory outputs that challenge existing non-discrimination laws. This paper analyzes these harms, categorizes them, and proposes legal and technical solutions to ensure fairer genAI development and deployment.

What Problem Does This Paper Address?

As generative AI (genAI) becomes increasingly integrated into various sectors, there's a growing concern about its potential to perpetuate and exacerbate existing discrimination. Unlike traditional AI, which might discriminate through biased data or algorithms leading to direct or indirect discrimination, genAI's communicative outputs (text, images, video) introduce new challenges. These outputs can be subtly biased through inadequate representation or overtly demeaning and abusive. Existing non-discrimination laws, primarily designed for tangible decisions and actions, struggle to adequately address these nuanced, often aggregate, harms. This paper aims to bridge this gap by analyzing genAI-driven discrimination through the lens of EU non-discrimination law and proposing reforms.

Methods

The paper employs a legal analysis approach, examining existing EU non-discrimination law and its applicability to genAI. It categorizes genAI's discriminatory outputs into three main types: discriminatory content, harassment, and "hard cases" of generative harm (inadequate representation, harmful stereotypes, misclassification). The authors then analyze how these categories map onto existing legal concepts like direct and indirect discrimination, and harassment, highlighting shortcomings. Finally, they propose policy options, including clarifying and updating laws, and shaping technology through regulation, testing, and auditing.

Datasets

Not discussed in the paper.

Key Insights and Ideas

  • Two main categories of genAI discrimination: The paper distinguishes between "demeaning and abusive content" (direct insults, hate speech) and "inadequate representation" (subtler biases through skewed representation over time).
  • "Hard cases" of generative harm: GenAI introduces novel challenges that don't neatly fit traditional legal categories. These include inadequate representation, harmful stereotypes, and misclassification, which can be insidious and difficult to prove.
  • Legal frameworks are lagging: Current non-discrimination laws, focused on tangible disadvantages and direct intent, are often ill-equipped to address the aggregate, subtle, and representational harms caused by genAI.
  • Joint and several liability is crucial: The authors argue that both developers and deployers of genAI systems should be held jointly and severally liable for discriminatory outputs to ensure accountability.
  • Technical solutions are necessary but not sufficient: While technical measures like bias mitigation in training data and auditing are important, they must be complemented by legal and regulatory changes.
  • The "Brown v. Board of Education" analogy: The paper draws inspiration from this landmark case to argue for legal recognition of intangible harms, such as feelings of inferiority or stigma, caused by genAI.

Key Results

The paper does not present empirical results but rather a legal and conceptual analysis. Its key findings are:

  • GenAI outputs can manifest discrimination in ways that are difficult to address with current legal frameworks.
  • Existing legal categories like direct/indirect discrimination and harassment are often inadequate for genAI-specific harms.
  • The AI Act, GDPR, and AI Liability Directives, as they currently stand, have limitations in addressing genAI discrimination.
  • The authors propose extending the scope of non-discrimination law to cover intangible harms and advocate for joint liability for developers and deployers.

Limitations and Future Work

The authors acknowledge that their analysis is primarily focused on EU law and that the application of these concepts outside the EU may vary. They also note that the legal landscape for AI is rapidly evolving, and future case law and legislative developments will be crucial. Future work suggested includes:

  • Further research into the types of content that should be displayed by genAI systems, considering both statistical reality and normative standards.
  • Developing clearer legal definitions and frameworks to address intangible harms caused by genAI.
  • Exploring novel legal concepts to better capture the unique forms of discrimination emerging from genAI.
  • Encouraging policymakers to clarify and update existing laws, and to shape genAI technology through regulations on testing, auditing, and inclusive content.

Summary updated

BibTeX
@misc{hacker2024generative,
  author = {Philipp Hacker and Brent D. Mittelstadt and Frederik J. Zuiderveen Borgesius and Sandra Wachter},
  title = {{Generative Discrimination: What Happens When Generative AI Exhibits Bias, and What Can Be Done About It}},
  year = {2024},
  eprint = {2407.10329},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2407.10329},
}
Figure