Retiring Adult: New Datasets for Fair Machine Learning

Frances Ding, Moritz Hardt, John Miller, Ludwig Schmidt

TL;DR

The algorithmic fairness community has long over-relied on the 1994 "UCI Adult" tabular dataset, but its arbitrary design choices—specifically a binary $50,000 income threshold—distort empirical findings and mask true algorithmic disparities. This paper exposes these flaws by reconstructing the original data and introduces folktables, a new Python package and suite of large-scale, multi-year, multi-state datasets from the US Census to provide a modern foundation for fair machine learning research.

What Problem Does This Paper Address?

In machine learning fairness research, tabular data benchmarks are surprisingly scarce. For over a decade, hundreds of papers have relied almost exclusively on the UCI Adult dataset to build, evaluate, and compare fairness interventions.

However, the authors reveal a major flaw: the dataset's target task is to predict whether an individual's income exceeds $50,000. In 1994, this threshold represented the 76th quantile of income overall, but the 88th quantile for Black individuals and the 89th quantile for women. Because the vast majority of these historically disadvantaged groups fall below this threshold, standard models achieve deceptively high accuracies on them simply by predicting "less than $50k." This idiosyncratic threshold understates and misrepresents how fairness interventions actually perform across the broader income distribution.

Fairness interventions and violations vary drastically depending on the chosen income threshold, showing that the standard $50k threshold exposes an extreme edge case.

Methods

The authors take a two-pronged approach:

  1. Archaeological Reconstruction: They trace the exact, previously undocumented source of the UCI Adult dataset back to the 1994 Current Population Survey (CPS) conducted by the US Census Bureau. They successfully reconstruct a closely matching superset called IPUMS Adult to analyze how changing the income threshold impacts fairness metrics.
  2. The folktables Ecosystem: To move the community forward, they tap into the American Community Survey (ACS) Public Use Microdata Sample (PUMS). They develop a Python package (folktables) that programmatically downloads, filters, and instantiates new, diverse prediction tasks across multiple domains, geographic locations, and timeframes.

Datasets

The paper introduces a suite of five new prediction tasks derived from modern US Census data (2014–2018), spanning all 50 states and Puerto Rico:

  • ACSIncome: Predict whether an individual's income is above $50,000 (designed as a direct, but cleaner and larger, replacement for UCI Adult).
  • ACSPublicCoverage: Predict whether a low-income individual has public health insurance coverage.
  • ACSMobility: Predict whether a young adult (ages 18–35) moved addresses in the last year.
  • ACSEmployment: Predict whether an adult is employed.
  • ACSTravelTime: Predict whether a working adult's commute is longer than 20 minutes.
Table 1: New prediction task details instantiated on 2018 US-wide ACS PUMS data

Key Insights and Ideas

  • Fairness Interventions are Region-Dependent: The effect size and success of a fairness intervention vary dramatically from state to state. A method that successfully mitigates demographic parity violations in California might have virtually no effect or cause massive accuracy drops in New York.
  • Geographic Transfer is Volatile: Models trained on one state and tested on another yield highly unpredictable results. Fairness constraints satisfied in the training state frequently fail when the model is deployed out-of-distribution in another state.
  • Temporal Stability is Surprisingly High: While predictive accuracy naturally degrades over time (e.g., a model trained on 2014 data evaluated on 2018 data), fairness criteria remain remarkably stable across years.
  • More Data Won't Fix Tabular Disparities: In cognitive tasks like computer vision, collecting more data can drive error rates toward zero and naturally mitigate group disparities. In tabular data, however, the "Bayes error rate" is inherently non-zero due to persistent systemic inequalities. Larger datasets simply reflect these base-rate disparities more precisely rather than resolving them.

Key Results

By varying the income threshold on their reconstructed dataset, the authors demonstrate that fairness violations peak near the median income (~$25,000) and drop off significantly at the $50,000 mark.

Furthermore, when evaluating geographic distribution shifts using the new folktables suite, the authors show the stark volatility of naive model deployment across states:

Evaluating models across different states shows that fairness interventions do not transfer gracefully, often failing to satisfy fairness constraints out-of-distribution.
Unconstrained and post-processed models evaluated out-of-distribution (OOD) across 49 other states show highly unpredictable accuracy and fairness trade-offs.

Limitations and Future Work

The primary limitation acknowledged by the authors is that their new dataset ecosystem is entirely US-centric. The fairness community still critically lacks diverse, high-quality tabular benchmarks representing international contexts, particularly from the Global South.

Additionally, the authors emphasize that these datasets are explicitly designed for benchmarking machine learning algorithms, not for drawing definitive sociological conclusions about real-world inequality or demographic shifts. They encourage the community to use folktables to explore new prediction tasks and further study the impacts of distribution shift on algorithmic fairness interventions.

Summary updated

Abstract

Although the fairness community has recognized the importance of data, researchers in the area primarily rely on UCI Adult when it comes to tabular data. Derived from a 1994 US Census survey, this dataset has appeared in hundreds of research papers where it served as the basis for the development and comparison of many algorithmic fairness interventions. We reconstruct a superset of the UCI Adult data from available US Census sources and reveal idiosyncrasies of the UCI Adult dataset that limit its external validity. Our primary contribution is a suite of new datasets derived from US Census surveys that extend the existing data ecosystem for research on fair machine learning. We create prediction tasks relating to income, employment, health, transportation, and housing. The data span multiple years and all states of the United States, allowing researchers to study temporal shift and geographic variation. We highlight a broad initial sweep of new empirical insights relating to trade-offs between fairness criteria, performance of algorithmic interventions, and the role of distribution shift based on our new datasets. Our findings inform ongoing debates, challenge some existing narratives, and point to future research directions. Our datasets are available at https://github.com/zykls/folktables.
BibTeX
@inproceedings{ding2021retiring,
  author = {Frances Ding and Moritz Hardt and John Miller and Ludwig Schmidt},
  title = {{Retiring Adult: New Datasets for Fair Machine Learning}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2021},
  url = {https://openreview.net/forum?id=bYi_2708mKK},
}
Figure