TL;DR
The algorithmic fairness community has long over-relied on the 1994 "UCI Adult" tabular dataset, but its arbitrary design choices—specifically a binary $50,000 income threshold—distort empirical findings and mask true algorithmic disparities. This paper exposes these flaws by reconstructing the original data and introduces folktables, a new Python package and suite of large-scale, multi-year, multi-state datasets from the US Census to provide a modern foundation for fair machine learning research.
What Problem Does This Paper Address?
In machine learning fairness research, tabular data benchmarks are surprisingly scarce. For over a decade, hundreds of papers have relied almost exclusively on the UCI Adult dataset to build, evaluate, and compare fairness interventions.
However, the authors reveal a major flaw: the dataset's target task is to predict whether an individual's income exceeds $50,000. In 1994, this threshold represented the 76th quantile of income overall, but the 88th quantile for Black individuals and the 89th quantile for women. Because the vast majority of these historically disadvantaged groups fall below this threshold, standard models achieve deceptively high accuracies on them simply by predicting "less than $50k." This idiosyncratic threshold understates and misrepresents how fairness interventions actually perform across the broader income distribution.
Methods
The authors take a two-pronged approach:
- Archaeological Reconstruction: They trace the exact, previously undocumented source of the UCI Adult dataset back to the 1994 Current Population Survey (CPS) conducted by the US Census Bureau. They successfully reconstruct a closely matching superset called IPUMS Adult to analyze how changing the income threshold impacts fairness metrics.
- The
folktablesEcosystem: To move the community forward, they tap into the American Community Survey (ACS) Public Use Microdata Sample (PUMS). They develop a Python package (folktables) that programmatically downloads, filters, and instantiates new, diverse prediction tasks across multiple domains, geographic locations, and timeframes.
Datasets
The paper introduces a suite of five new prediction tasks derived from modern US Census data (2014–2018), spanning all 50 states and Puerto Rico:
- ACSIncome: Predict whether an individual's income is above $50,000 (designed as a direct, but cleaner and larger, replacement for UCI Adult).
- ACSPublicCoverage: Predict whether a low-income individual has public health insurance coverage.
- ACSMobility: Predict whether a young adult (ages 18–35) moved addresses in the last year.
- ACSEmployment: Predict whether an adult is employed.
- ACSTravelTime: Predict whether a working adult's commute is longer than 20 minutes.
Key Insights and Ideas
- Fairness Interventions are Region-Dependent: The effect size and success of a fairness intervention vary dramatically from state to state. A method that successfully mitigates demographic parity violations in California might have virtually no effect or cause massive accuracy drops in New York.
- Geographic Transfer is Volatile: Models trained on one state and tested on another yield highly unpredictable results. Fairness constraints satisfied in the training state frequently fail when the model is deployed out-of-distribution in another state.
- Temporal Stability is Surprisingly High: While predictive accuracy naturally degrades over time (e.g., a model trained on 2014 data evaluated on 2018 data), fairness criteria remain remarkably stable across years.
- More Data Won't Fix Tabular Disparities: In cognitive tasks like computer vision, collecting more data can drive error rates toward zero and naturally mitigate group disparities. In tabular data, however, the "Bayes error rate" is inherently non-zero due to persistent systemic inequalities. Larger datasets simply reflect these base-rate disparities more precisely rather than resolving them.
Key Results
By varying the income threshold on their reconstructed dataset, the authors demonstrate that fairness violations peak near the median income (~$25,000) and drop off significantly at the $50,000 mark.
Furthermore, when evaluating geographic distribution shifts using the new folktables suite, the authors show the stark volatility of naive model deployment across states:
Limitations and Future Work
The primary limitation acknowledged by the authors is that their new dataset ecosystem is entirely US-centric. The fairness community still critically lacks diverse, high-quality tabular benchmarks representing international contexts, particularly from the Global South.
Additionally, the authors emphasize that these datasets are explicitly designed for benchmarking machine learning algorithms, not for drawing definitive sociological conclusions about real-world inequality or demographic shifts. They encourage the community to use folktables to explore new prediction tasks and further study the impacts of distribution shift on algorithmic fairness interventions.
Summary updated