When we evaluate a machine learning model on a "novel" downstream task, we operate under a fundamental assumption: the evaluation data represents a domain or a set of categories that the model has never encountered before. This assumption is critical whether we are testing a model's ability to generalize from a few examples (few-shot learning) or its ability to sequentially acquire new knowledge without forgetting the old (continual learning).
However, the rise of web-scale pre-training has quietly invalidated this assumption. Because modern models are trained on massive, uncurated datasets, the "unseen" categories used in downstream benchmarks are frequently hidden within the model's pre-training corpus. This phenomenon—known variously as pre-training data leakage, category leakage, or implicit shot bias—creates a powerful illusion. It masks a model's true generalization and learning capabilities, turning tests of adaptability into mere exercises in memory retrieval.
The Illusion of the Unseen Task
To understand why pre-training data leakage is so problematic, consider the standard protocol for few-shot learning. A feature extractor is pre-trained on a base dataset, and then a classifier is tasked with learning a set of novel target classes using only one or two examples.
As Anish Madan et al. demonstrate in Revisiting Few-Shot Object Detection with Vision-Language Models, this protocol is fundamentally broken when applied to modern foundation models. Because these models have already encountered web-scale representations of supposedly "novel" concepts, the evaluation no longer measures how effectively a model learns from scratch. Instead, it measures how well the model aligns its pre-existing representations to a specific prompt or context.
When a model has already seen a concept thousands of times during pre-training, its high performance on a few-shot benchmark does not reflect robust few-shot generalization. As Peiyuan Zhang et al. point out in Better Few-Shot Relation Extraction with Label Prompt Dropout, this overlap creates an artificial boost in performance metrics. The model relies heavily on textual labels and prior exposure rather than learning robust, task-specific semantic representations from the limited data provided.
Implicit Shots and Category Leakage
This evaluation failure manifests clearly in computer vision tasks like object detection and action recognition. When the boundaries between pre-training data and evaluation data blur, models benefit from what Chenchen Zhu et al. term implicit shot bias in Semantic Relation Reasoning for Shot-Stable Few-Shot Object Detection.
Standard Evaluation Assumption:
[Pre-training: Base Classes] ──> [Few-Shot Evaluation: Truly Novel Classes]
The Reality of Data Contamination:
[Pre-training: Base + "Novel" Classes (Implicit Shots)] ──> [Evaluation: "Novel" Classes (Memory Retrieval)]
When these implicit examples are strictly filtered out, standard visual models experience severe performance degradation. This drop reveals that the models were not actually adapting to real-world scarcity; they were simply recalling prior knowledge.
Similarly, in the domain of video analysis, Yichao Cao et al. explore this as category leakage in Searching for Better Spatio-temporal Alignment in Few-Shot Action Recognition. When a pre-trained backbone inadvertently learns the features of evaluation classes beforehand, it becomes impossible for researchers to distinguish between a model's genuine few-shot learning capability and its capacity to recall previously seen temporal patterns.
Masking Catastrophic Forgetting
The consequences of data leakage extend far beyond few-shot learning, severely compromising how we evaluate continual learning (CL) and class-incremental learning (CIL). The core challenge of continual learning is overcoming catastrophic forgetting—the tendency of a model to overwrite old knowledge when trained on new tasks.
However, if the downstream evaluation tasks overlap with the model's initial pre-training data, the true extent of forgetting is obscured. In DESIRE: Dynamic Knowledge Consolidation for Rehearsal-Free Continual Learning, Fei Zhu et al. explain that evaluating models on contaminated datasets makes it difficult to determine whether a framework is genuinely retaining new information or simply falling back on deeply ingrained pre-trained representations.
This issue is quantified by Çağatay Yıldız et al. in Investigating Continual Pretraining in Large Language Models: Insights and Implications, who show that overlapping content directly masks the loss of knowledge during sequential domain updates. Because the model has already memorized the evaluation concepts globally, standard metrics fail to register the degradation of the model's sequential learning capacity.
This vulnerability spans across modalities:
- Image Classification: Da-Wei Zhou et al. demonstrate in PILOT: a pre-trained model-based continual learning toolbox that standard image benchmarks yield highly misleading results if the pre-trained models were originally exposed to those same datasets.
- Multimodal Instruction Tuning: As shown by Fei Zhu et al. in HiDe-LLaVA: Hierarchical Decoupling for Continual Instruction Tuning of Multimodal Large Language Model, information leakage from initial supervised fine-tuning data masks the true extent of forgetting in multimodal large language models, making them appear more stable than they actually are.
Designing Leak-Free Evaluations
To accurately measure machine learning progress, the research community is shifting toward evaluation methodologies that explicitly account for pre-training distributions.
One approach is to enforce strict data isolation during benchmark creation. In A New Split for Evaluating True Zero-Shot Action Recognition, Shreyank N Gowda et al. introduce the "TruZe" (True Zero-Shot) split. By systematically identifying and removing overlapping classes from the pre-training scope, the authors reset the baseline for action recognition, forcing models to perform true zero-shot generalization without a safety net of leaked data.
Another strategy involves shifting benchmarks toward significant domain gaps rather than simple class differences. As argued by Da-Wei Zhou et al. in Revisiting Class-Incremental Learning with Pre-Trained Models: Generalizability and Adaptivity are All You Need, because frozen pre-trained embeddings are incredibly robust, traditional class-incremental benchmarks are no longer sufficient. Evaluating the true balance between generalizability and adaptivity requires testing models on entirely new domains that force the architecture to update its underlying representations, rather than merely remapping its pre-existing memory.
Key Foundational Work
Semantic Relation Reasoning for Shot-Stable Few-Shot Object Detection (2021) — contributed 3 concept elements to this direction.
A New Split for Evaluating True Zero-Shot Action Recognition (2021) — contributed 2 concept elements to this direction.
Searching for Better Spatio-temporal Alignment in Few-Shot Action Recognition (2022) — contributed 2 concept elements to this direction.