No article text was recorded for this version.
training data deduplication
Version 1 of 135 papers44 conceptsfrozen 2026-10-04
Papers in this version
- Blog: Blameless Users in a Clean Room: Defining Copyright Protection for Generative Models
- Blog: Dive into the Chasm: Probing the Gap between In- and Cross-Topic Generalization
- Blog: Deduplicating Training Data Mitigates Privacy Risks in Language Models
- Blog: A Survey on Data Selection for Language Models
- Blog: Generation or Replication: Auscultating Audio Latent Diffusion Models
- Blog: Deduplicating Training Data Mitigates Privacy Risks in Language Models
- Blog: Provably Confidential Language Modelling
- Blog: Yankari: Monolingual Yoruba Dataset
- Blog: Investigating Memorization in Video Diffusion Models
- Blog: Deduplicating Training Data Makes Language Models Better
- Blog: Finding Memo: Extractive Memorization in Constrained Sequence Generation Tasks
- Blog: Introducing the Enron Corpus
- Blog: GPT-NeoX-20B: An Open-Source Autoregressive Language Model
- Blog: Measuring memorization in language models via probabilistic extraction
- Blog: Quantifying and Analyzing Entity-Level Memorization in Large Language Models
- Blog: Emergent and Predictable Memorization in Large Language Models
- Blog: Counterfactual Memorization in Neural Language Models
- Blog: Big Code != Big Vocabulary: Open-Vocabulary Models for Source Code
- Blog: What Do Code Models Memorize? An Empirical Study on Large Language Models of Code
- Blog: Counterfactual Influence as a Distributional Quantity
- Blog: Language Generation Models Can Cause Harm: So What Can We Do About It? An Actionable Survey
- Blog: Exploring Memorization and Copyright Violation in Frontier LLMs: A Study of the New York Times v. OpenAI 2023 Lawsuit
- Blog: Copyright Traps for Large Language Models
- Blog: A Pathway Towards Responsible AI Generated Content
- Blog: An Integrated Data Processing Framework for Pretraining Foundation Models
- Blog: Training Data Extraction From Pre-trained Language Models: A Survey
- Blog: Data Contamination: From Memorization to Exploitation
- Blog: Duplicate entries in the Protein Data Bank: how to detect and handle them
- Blog: Assessing Privacy Risks in Language Models: A Case Study on Summarization Tasks
- Blog: On the evaluation of neural code summarization
- Blog: Just Fine-tune Twice: Selective Differential Privacy for Large Language Models
- Blog: PoisonedParrot: Subtle Data Poisoning Attacks to Elicit Copyright-Infringing Content from Large Language Models
- Blog: How do Categorical Duplicates Affect ML? A New Benchmark and Empirical Analyses
- Blog: A Review of the Challenges with Massive Web-mined Corpora Used in Large Language Models Pre-Training
- Blog: The RefinedWeb Dataset for Falcon LLM: Outperforming Curated Corpora with Web Data Only
History & citation
Current · Permanent link
- This direction split from Data Duplication and the Mechanics of Model Memorization (69% overlap)
Cite this direction
Latest version (a living page):
@misc{lacuna_training_data_deduplication_de7560f6,
title = {training data deduplication},
author = {{Lacuna}},
howpublished = {Lacuna research direction},
year = {2026},
url = {https://lacuna.tiptreesystems.com/direction/training-data-deduplication/rdr_de7560f6a6e44b0ba022f038b6326a78},
note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-22},
}This exact version (version 1, frozen text):
@misc{lacuna_training_data_deduplication_de7560f6_v1,
title = {training data deduplication},
author = {{Lacuna}},
howpublished = {Lacuna research direction},
year = {2026},
url = {https://lacuna.tiptreesystems.com/direction/training-data-deduplication/rdr_de7560f6a6e44b0ba022f038b6326a78/v/1},
note = {Version 1, frozen 2026-10-04. Accessed 2026-11-22},
}