TL;DR
This paper provides the first in-depth documentation and analysis of the Colossal Clean Crawled Corpus (C4), a massive dataset used for training large language models. The authors reveal surprising data sources, identify issues like machine-generated text and benchmark contamination, and crucially, demonstrate that the dataset's filtering process disproportionately removes content related to minority groups.
What Problem Does This Paper Address?
The advent of large language models (LLMs) has led to a demand for ever-larger training datasets, often scraped from the internet. However, these massive web-scraped corpora are frequently released with minimal documentation, leaving researchers and practitioners in the dark about their origins, content, and potential biases. This lack of transparency hinders reproducible science and can lead to unintended consequences when LLMs are deployed. This paper addresses this critical gap by providing a comprehensive analysis of the C4 dataset, a prominent example of such a corpus.
Methods
The authors employ a multi-faceted approach to document and analyze the C4 dataset:
- Provenance Analysis: They investigate the origins of the data by examining top-level domains (TLDs) and specific websites, identifying unexpected sources like patent databases and US military sites.
- Content Analysis: They analyze the text itself to identify machine-generated content (e.g., from machine translation) and evidence of "data contamination," where examples from existing NLP benchmarks appear in the training data.
- Filter Impact Analysis: A key contribution is evaluating the effect of the filtering process used to create C4. They specifically examine the impact of a "blocklist" filter designed to remove offensive language, analyzing which types of documents are disproportionately removed.
- Bias Measurement: They reproduce and extend previous work on measuring demographic biases, specifically focusing on ethnicity and sentiment.
Datasets
- Colossal Clean Crawled Corpus (C4): The primary dataset analyzed. The authors provide three versions:
- C4.EN.NOCLEAN: The snapshot of Common Crawl identified as English, with no additional filters.
- C4.EN.NOBLOCKLIST: C4.EN without the "bad words" blocklist filter.
- C4.EN: The fully cleaned version of the corpus.
- Existing NLP Benchmarks: The paper examines contamination from various NLP benchmarks, including:
- Abstractive summarization datasets (TIFU, XSum)
- Table-to-text generation (WikiBio)
- Graph-to-text generation (AMR-to-text)
- LAMA dataset (for probing knowledge completion)
- GLUE benchmark (for general language understanding)
Key Insights and Ideas
- Unexpected Data Sources: C4 contains a significant amount of text from sources like patent databases (e.g., patents.google.com) and US military websites, which are not typical for general language modeling.
- Machine-Generated Content is Prevalent: The dataset includes substantial amounts of machine-generated text, such as from machine translation systems, highlighting the increasing presence of synthetic data on the web.
- Benchmark Contamination is Real: Existing NLP benchmark examples are found in C4, sometimes at surprisingly high rates (e.g., over 50% for QNLI sentence inputs). This can lead to inflated performance metrics and an inaccurate understanding of model capabilities.
- Blocklist Filtering is Problematic: The "bad words" blocklist, intended to remove offensive content, disproportionately filters out documents related to minority identities, including sexual orientations and African American English. This is a form of "allocational harm."
- Bias is Embedded and Amplified: The paper provides evidence that C4 exhibits sentiment bias, with "Jewish" being associated with more positive sentiment than "Arab." This bias likely stems from the web data and can be amplified by LLMs trained on it.
Key Results
- Domain Distribution: The top represented TLDs in C4.EN are .com, .org, and .net. Notably, .gov and .mil domains are also significant sources.
- Website Representation: patents.google.com is the most represented website, followed by Wikipedia and news outlets.
- Benchmark Overlap: Target texts from datasets like XSum and TIFU-short appear in C4.EN at rates of 15.49% and 24.88%, respectively. For GLUE, QNLI sentence inputs show over 53% overlap.
- Filter Exclusion: African American English (AAE) and Hispanic-aligned English are removed at significantly higher rates (42% and 32%) by the blocklist filter compared to White-aligned English (6.2%).
- Sentiment Bias: The study found a positivity gap of 7.5% between "Jewish" and "Arab" in C4.EN, with "Jewish" having a higher proportion of positive sentiment.
Limitations and Future Work
The authors acknowledge several limitations:
- Internet Archive Proxy: Using the Internet Archive's indexing date as a proxy for utterance date is not perfect.
- Geolocation Imperfections: IP-based geolocation has inherent limitations.
- Causality of Bias: While they show correlation, they haven't definitively proven a causal link between C4's biases and downstream model behavior.
- Completeness of Analysis: They recognize that only a subset of potential issues (like personally identifiable information or copyrighted text) were examined.
- Language Generalization: The analyses are primarily focused on English and may not generalize to other languages.
Future work could involve developing more nuanced filtering mechanisms beyond simple blocklists, creating more transparent and auditable datasets, and further investigating the causal links between dataset biases and model behavior. The authors also recommend providing a mechanism for reporting issues found in the dataset.
Summary updated