TL;DR
This paper tackles the critical but under-explored issue of personal information (PI) in large language model training data. The authors propose a novel PI taxonomy, analyze its prevalence in major datasets like C4 and The Pile, and compare detection methods, revealing significant risks and limitations in current approaches.
What Problem Does This Paper Address?
Large language models (LLMs) are increasingly trained on massive web-scraped datasets. While efforts have been made to filter out toxic or biased content, the presence and detection of personal information (PI) in these training corpora remain largely unaddressed. This is a significant concern because LLMs can inadvertently memorize and then generate PI, posing risks of identity theft, discrimination, and exposure to harmful situations for individuals. The paper aims to formally define PI, categorize its risks, and evaluate current methods for its detection in large-scale datasets.
Methods
The authors first establish a novel PI taxonomy, categorizing PI by type (birth-centered, society-centered, social-based, character-based, records-based, situation-based) and risk level (low, medium, high, extreme). They then conduct a case study on two large datasets, the Colossal Clean Crawled Corpus (C4) and The Pile, focusing on "character-based" PI due to its high risk. For detection, they compare a model-based tool, Presidio, with a traditional regular-expression (regex) based approach. Their evaluation involves both quantitative counting of detected PI instances and a manual audit of a random sample of detections to assess accuracy.
Datasets
- Colossal Clean Crawled Corpus (C4): A large language dataset comprising over 365 million documents and 173 billion tokens, derived from Common Crawl and filtered for high-quality English text.
- The Pile: A composite English dataset of 22 smaller datasets, totaling 383 billion tokens, covering diverse genres like science, law, and literature.
Key Insights and Ideas
- A Novel PI Taxonomy is Crucial: Existing definitions of PI are vague and vary by region. A structured taxonomy, like the one proposed, is essential for understanding PI's diverse forms and associated risks.
- Character-Based PI is High-Risk: Elements like credit card numbers, IBANs, and email addresses are particularly risky because they can uniquely identify individuals and are often used for financial or identity-related activities.
- Current Detection Tools Fall Short: While some PI types (like email addresses) are well-detected, others, especially high-risk ones like SSNs and credit card numbers, are poorly captured by both Presidio and regex-based methods, leading to significant false positives and negatives.
- Regexes Can Outperform ML for Some PI: For certain PI types with rigid syntax, regexes can be more effective than ML-based tools like Presidio, though they can also be prone to false positives if not carefully crafted.
- Linked PI Amplifies Risk: When different types of PI appear close together in a text, the risk of uniquely identifying an individual increases dramatically, echoing findings that even quasi-identifiers can be combined to de-anonymize individuals.
Key Results
- Prevalence of PI: Both C4 and The Pile contain millions of instances of personal information.
- Detection Accuracy Varies:
- High Accuracy: Phone numbers and email addresses showed high detection accuracy (in the 90s) for both Presidio and regexes.
- Moderate Accuracy: IBAN codes were well-detected by Presidio (98.5%) but poorly by regexes (15.2%). IP addresses showed better detection by regexes (49.8%) than Presidio (25.9%).
- Low Accuracy: US bank numbers, US social security numbers, and credit cards had significantly lower detection accuracy for both methods, often resulting in false positives.
- Presidio vs. Regex: Regexes systematically captured more instances of PI than Presidio for phone numbers, US SSNs, credit cards, and IBAN codes. Presidio detected more IP addresses. For names and US bank numbers, regexes were not developed, so direct comparison was not possible.
- Linked PI: In C4, less than 2.7% of detected PI instances were other types besides names, but nearly 2.5% had at least one other type of PI in close proximity, indicating increased risk. This was exacerbated in The Pile.
Limitations and Future Work
- Limitations:
- Ground Truth: The study lacked exhaustive PI ground truth annotations, making it impossible to measure false negatives and fully characterize precision and recall.
- Language/Country Specificity: Detection approaches are often language and country-specific, requiring adaptation for different contexts.
- New PI Forms: The study acknowledges that new forms of PI are emerging with the internet (e.g., Facebook event URLs) that were not covered by traditional definitions.
- Future Work:
- Develop Better Detection Tools: There's a need for more robust, modular, and extensive PI detection tools, potentially combining ML and rule-based approaches.
- Improve PI Taxonomy: Further refinement and expansion of the PI taxonomy are needed.
- Practicing Consent: Emphasizing ethical data collection practices, including explicit consent from individuals, is crucial.
- Detecting PI Memorization: Developing systematic methods to test trained models for PI memorization is vital, especially for sensitive applications.
- Synthetic Data: The authors are working on methods to develop synthetic, lower-risk labeled corpora to aid in developing better PI detection methods.
Summary updated