No article text was recorded for this version.
benchmark data contamination
Version 2 of 23,764 papers5,466 conceptsfrozen 2026-10-04
Papers in this version
- Blog: Proceedings of the 1st Workshop on Data Contamination (CONDA)
- Blog: Don't Make Your LLM an Evaluation Benchmark Cheater
- Blog: Evaluation of LLMs in Speech is Often Flawed: Test Set Contamination in Large Language Models for Speech Recognition
- Blog: NLP Evaluation in trouble: On the Need to Measure LLM Data Contamination for each Benchmark
- Blog: A Careful Examination of Large Language Model Performance on Grade School Arithmetic
- Blog: Investigating Data Contamination for Pre-training Language Models
- Blog: Evaluation of LLMs in Speech is Often Flawed: Test Set Contamination in Large Language Models for Speech Recognition
- Blog: Data Contamination in LLMs: A Scoping Literature Review
- Blog: The Impact of Post-training on Data Contamination
- Blog: Can we trust the evaluation on ChatGPT?
- Blog: Are Large Language Models Memorizing Bug Benchmarks?
- Blog: Reasoning or Memorization? Unreliable Results of Reinforcement Learning Due to Data Contamination
- Blog: The Tale of Two MSMARCO - and Their Unfair Comparisons
- Blog: Simulating Training Data Leakage in Multiple-Choice Benchmarks for LLM Evaluation
- Blog: Estimating Contamination via Perplexity: Quantifying Memorisation in Language Model Evaluation
- Blog: Stop Uploading Test Data in Plain Text: Practical Strategies for Mitigating Data Contamination by Evaluation Benchmarks
- Blog: LessLeak-Bench: A First Investigation of Data Leakage in LLMs Across 83 Software Engineering Benchmarks
- Blog: Leak, Cheat, Repeat: Data Contamination and Evaluation Malpractices in Closed-Source LLMs
- Blog: The Emperor's New Clothes in Benchmarking? A Rigorous Examination of Mitigation Strategies for LLM Benchmark Data Contamination
- Blog: Data Contamination Report from the 2024 CONDA Shared Task
- Blog: Pretraining on the Test Set Is All You Need
- Blog: Stop Uploading Test Data in Plain Text: Practical Strategies for Mitigating Data Contamination by Evaluation Benchmarks
- Blog: The Adverse Effects of Code Duplication in Machine Learning Models of\n Code
- Blog: Koala: An Index for Quantifying Overlaps with Pre-training Corpora
- Blog: The GitHub Recent Bugs Dataset for Evaluating LLM-based Debugging Applications
- Blog: CLEAN-EVAL: Clean Evaluation on Contaminated Large Language Models
- Blog: DCR: Quantifying Data Contamination in LLMs Evaluation
- Blog: Benchmark Inflation: Revealing LLM Performance Gaps Using Retro-Holdouts
- Blog: On Train-Test Class Overlap and Detection for Image Retrieval
- Blog: Quantifying Contamination in Evaluating Code Generation Capabilities of Language Models
- Blog: Question and Answer Test-Train Overlap in Open-Domain Question Answering Datasets
- Blog: CAP: Data Contamination Detection via Consistency Amplification
- Blog: Missing Counter-Evidence Renders NLP Fact-Checking Unrealistic for Misinformation
- Blog: Beyond Memorization: Reasoning-Driven Synthesis as a Mitigation Strategy Against Benchmark Contamination
- Blog: Evaluating Chinese Large Language Models on Discipline Knowledge Acquisition via Memorization and Robustness Assessment
- Blog: Rethinking the effects of data contamination in Code Intelligence
- Blog: LastingBench: Defend Benchmarks Against Knowledge Leakage
- Blog: C²LEVA: Toward Comprehensive and Contamination-Free Language Model Evaluation
- Blog: On Train-Test Class Overlap and Detection for Image Retrieval
- Blog: An Analysis of Dataset Overlap on Winograd-Style Tasks
- Blog: To the Cutoff... and Beyond? A Longitudinal Perspective on LLM Data Contamination
- Blog: Benchmark Probing: Investigating Data Leakage in Large Language Models
- Blog: Avoiding Overlap in Data Augmentation for AMR-to-Text Generation
- Blog: Language Models are Surprisingly Fragile to Drug Names in Biomedical Benchmarks
- Blog: Assessing Contamination in Large Language Models: Introducing the LogProber method
- Blog: Language-Models-as-a-Service: Overview of a New Paradigm and its Challenges
- Blog: On Inter-dataset Code Duplication and Data Leakage in Large Language Models
- Blog: A Taxonomy for Data Contamination in Large Language Models
- Blog: PLINDER: The protein-ligand interactions dataset and evaluation resource
- Blog: Quantifying Data Contamination in Psychometric Evaluations of LLMs
- Blog: ChatGPT Participates in a Computer Science Exam
- Blog: A Taxonomy for Data Contamination in Large Language Models
- Blog: CLEAN-EVAL: Clean Evaluation on Contaminated Large Language Models
- Blog: On Leakage of Code Generation Evaluation Datasets
- Blog: Enhancing Data Quality through Simple De-duplication: Navigating Responsible Computational Social Science Research
- Blog: Quantifying Data Contamination in Psychometric Evaluations of LLMs
- Blog: Does Data Contamination Make a Difference? Insights from Intentionally Contaminating Pre-training Data For Language Models
- Blog: Towards Data Contamination Detection for Modern Large Language Models: Limitations, Inconsistencies, and Oracle Challenges
- Blog: Don't Make Your LLM an Evaluation Benchmark Cheater
- Blog: Benchmarking LLMs for Unit Test Generation from Real-World Functions
Showing 60 of 3,764 papers.
History & citation
Current · Permanent link
- The Data Contamination Crisis in Zero-Shot Evaluation merged into this direction (94% overlap)
- Disentangling Reasoning from Memorization in Large Language Models merged into this direction (51% overlap)
- Data Contamination in Large Language Model Evaluation merged into this direction (100% overlap)
- Dynamic Evaluation to Mitigate Large Language Model Contamination merged into this direction (78% overlap)
- Dynamic Evaluation of Reasoning Beyond Training Data Cutoffs merged into this direction (68% overlap)
- Bridging the Gap Between Ethical Knowledge and Generative Behavior merged into this direction (57% overlap)
- pre-training data leakage split from this direction (50% overlap)
- dynamic benchmark generation split from this direction (65% overlap)
- dynamic evaluation overhead split from this direction (50% overlap)
Cite this direction
Latest version (a living page):
@misc{lacuna_benchmark_data_contamination_6b66948d,
title = {benchmark data contamination},
author = {{Lacuna}},
howpublished = {Lacuna research direction},
year = {2026},
url = {https://lacuna.tiptreesystems.com/direction/benchmark-data-contamination/rdr_6b66948d6aa54c30b19faec5e2672ea9},
note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-21},
}This exact version (version 2, frozen text):
@misc{lacuna_benchmark_data_contamination_6b66948d_v2,
title = {benchmark data contamination},
author = {{Lacuna}},
howpublished = {Lacuna research direction},
year = {2026},
url = {https://lacuna.tiptreesystems.com/direction/benchmark-data-contamination/rdr_6b66948d6aa54c30b19faec5e2672ea9/v/2},
note = {Version 2, frozen 2026-10-04. Accessed 2026-11-21},
}