A Canary in the AI Coal Mine: American Jews May Be Disproportionately Harmed by Intellectual Property Dispossession in Large Language Model Training

Heila Precel, Allison McDonald, Brent Hecht, Nicholas VincentarXiv 2024 · DOI · arXiv

TL;DR

Current large language model (LLM) training paradigms rely on massive, non-consensual data scraping, which this paper reveals disproportionately impacts American Jews due to their high representation in intellectual-property-heavy professions. Using demographic inference on open training datasets, the authors find that Jewish authors are over-represented by 2x to 6.5x, warning that legal "fair use" exemptions for AI training could institutionalize systemic economic bias against specific minority groups.

What Problem Does This Paper Address?

LLMs are trained on vast amounts of digital content without the explicit consent of or compensation to the original creators. Because these models are explicitly designed to automate tasks and potentially substitute for human labor, this data ingestion translates directly into economic risk for content creators.

The authors investigate whether this burden of "intellectual property dispossession" falls equally across demographic lines. Using American Jews as a case study—a group with a long historical context of physical and intellectual property seizure—the paper explores how structural factors like educational attainment and professional distribution make certain minority communities uniquely vulnerable to AI-driven labor substitution.

Methods

The authors adapt a probabilistic technique from Jewish demographic literature called the Distinctive Jewish Names (DJN) frame. Because the U.S. Census does not collect religious data, demographers use a curated list of 35 surnames that are highly specific to the Jewish community to estimate population metrics.

The authors apply this to LLM datasets via a five-step pipeline:

  1. Extract author names from dataset metadata.
  2. Match names against the 35-surname DJN list.
  3. Adjust for false positives (assuming an 80–90% precision rate based on historical literature).
  4. Adjust for population coverage (extrapolating from the fact that roughly 9.15–11.18% of U.S. Jews carry these specific names).
  5. Calculate a Relative Dispossession Magnitude—the ratio of observed to expected authorship—while controlling for the geographic distribution (U.S. vs. international) of the data.

Datasets

The study analyzes five high-quality, text-heavy components of The Pile (an open-source dataset curated by EleutherAI widely used to train LLMs), which together comprise 49.14% of the dataset by weight:

  • PubMed Central: Biomedical literature.
  • Books3: A corpus of digitized books.
  • ArXiv: Scientific preprints.
  • GitHub: Public code repositories (analyzed via a random 5% sample).
  • FreeLaw: U.S. court opinions and legal data.

Raw web crawls (like Pile-CC) were excluded because they lack structured author metadata.

Key Insights and Ideas

  • The "Canary in the Coal Mine" Effect: The disproportionate exposure of one minority group is not an isolated anomaly; it serves as an early warning sign of deep structural flaws in the current AI data paradigm that threaten to disrupt white-collar labor markets across the board.
  • The Education-Vulnerability Paradox: High educational attainment and participation in knowledge-producing fields (like law, medicine, and STEM) ironically make a group's livelihood more vulnerable to being scraped and replicated by LLMs under the current paradigm.
  • Legal and Policy Risks: If U.S. courts rule that training LLMs on copyrighted data constitutes "fair use," they may inadvertently codify a systemic material bias that economically harms specific minority groups in the short term.

Key Results

  • Across all analyzed datasets, content authored by Jewish Americans is substantially over-represented, ranging from 2.02x to 6.46x compared to their baseline proportion in the U.S. population.
  • The highest over-representation was found in GitHub (3.53x–6.46x) and ArXiv (3.07x–5.63x), reflecting high participation rates in STEM fields.
  • Even the most conservative lower-bound estimate across the entire weighted dataset shows a 2.46x over-representation, proving a clear, systemic material bias.
Dataset Relative Dispossession Magnitude
PubMed Central 2.02 - 3.71 X
Books3 2.92 - 5.36 X
ArXiv 3.07 - 5.63 X
GitHub 3.53 - 6.46 X
FreeLaw 2.77 - 5.08 X
Weighted Total 2.46 - 4.51 X

Limitations and Future Work

  • Name-Based Proxy Limitations: The DJN method inherently underrepresents certain subgroups, such as younger generations, intermarried individuals, or Jews without traditional surnames.
  • The Transparency Deficit: Because commercial AI giants (like OpenAI and Meta) keep their training data secret, the authors had to rely on open datasets like The Pile as a proxy for industry standards.
  • Future Directions: The authors call for replicating this framework for other minority groups with high educational attainment (e.g., Asian Americans, Hindu Americans). They also urge the computer science community to pivot toward alternative AI architectures, such as retrieval-augmented generation (RAG) or "data dividends" that compensate creators and allow for data opt-outs.

Summary updated

Abstract

Systemic property dispossession from minority groups has often been carried out in the name of technological progress. In this paper, we identify evidence that the current paradigm of large language models (LLMs) likely continues this long history. Examining common LLM training datasets, we find that a disproportionate amount of content authored by Jewish Americans is used for training without their consent. The degree of over-representation ranges from around 2x to around 6.5x. Given that LLMs may substitute for the paid labor of those who produced their training data, they have the potential to cause even more substantial and disproportionate economic harm to Jewish Americans in the coming years. This paper focuses on Jewish Americans as a case study, but it is probable that other minority communities (e.g., Asian Americans, Hindu Americans) may be similarly affected and, most importantly, the results should likely be interpreted as a "canary in the coal mine" that highlights deep structural concerns about the current LLM paradigm whose harms could soon affect nearly everyone. We discuss the implications of these results for the policymakers thinking about how to regulate LLMs as well as for those in the AI field who are working to advance LLMs. Our findings stress the importance of working together towards alternative LLM paradigms that avoid both disparate impacts and widespread societal harms.
BibTeX
@misc{precel2024canary,
  author = {Heila Precel and Allison McDonald and Brent Hecht and Nicholas Vincent},
  title = {{A Canary in the AI Coal Mine: American Jews May Be Disproportionately Harmed by Intellectual Property Dispossession in Large Language Model Training}},
  year = {2024},
  eprint = {2403.13073},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2403.13073},
}
Figure