Learning to Learn from Corrupted Data for Few-Shot Learning

Yuexuan An, Xingyu Zhao, Hui Xue

Introduction

Few-shot learning (FSL) aims to recognize new objects from a limited number of examples. While most FSL algorithms assume a pristine "base" dataset for pre-training, real-world data is often "dirty," containing feature noise from transmission or label noise from human error. FSL models are particularly sensitive to such corruption, often overfitting to noise rather than learning generalizable patterns. This paper introduces Peer-Collaboration Learning (PCL), a model-agnostic framework designed to filter out corrupted data and enhance robustness in noisy scenarios.

Methodology: Peer-Collaboration Learning (PCL)

The PCL framework is inspired by collaborative learning, where two interactive modules cross-examine training data to identify and down-weight problematic samples.

Overview of the Peer-Collaboration Learning (PCL) framework showing the interaction between two modules for sample importance extraction.

Architecture and Workflow

PCL consists of two modules that share a common feature extractor but utilize independent classification and importance-extraction heads. The process involves three key stages:

  1. Mutual Evaluation: Module A evaluates a batch and identifies "difficult" samples (high loss) for Module B, and vice-versa.
  2. Dual-Perspective Weighting: Sample importance is determined through two lenses:
  3. Global Weighting: A BiLSTM network ranks samples by loss values to identify global outliers.
  4. Local Weighting: A Transformer layer analyzes intra-class similarity to detect samples that deviate from the class prototype.
  5. Re-weighted Training: The calculated weights scale the loss of each sample, effectively filtering out corrupted data from gradient updates.

Experimental Evaluation

The framework was evaluated on four benchmarks: CIFAR-FS, CUB, mini-ImageNet, and tiered-ImageNet. To simulate real-world conditions, feature noise (random pixels) and label noise (mislabeled images) were injected at rates of 20% and 40%.

Key Results

PCL demonstrated significant performance gains across all tested baseline algorithms (MatchingNet, ProtoNet, S2M2, and DeepEMD). For instance, in 40% label noise scenarios on mini-ImageNet, PCL improved ProtoNet's accuracy by +5.27%.

Comparison of PCL-ProtoNet against other noise-robust methods (e.g., Co-teaching, JoCoR) across different noise types and rates.

Beyond noisy data, PCL also improved performance on clean datasets, suggesting it helps models focus on the most informative examples by filtering out inherent dataset ambiguities. Ablation studies confirmed that both global and local weighting are essential; global weighting excels at catching extreme noise, while local weighting is more effective for subtle mislabeling.

Conclusion and Future Work

PCL offers a robust, model-agnostic solution for few-shot learning under data corruption. While it introduces some computational overhead due to the dual-module architecture, its ability to generalize across various noise types and base algorithms is a significant advancement. Future research may focus on optimizing training efficiency and exploring deeper theoretical foundations for peer-collaboration in low-data regimes.

Summary updated

Abstract

Few-shot learning which aims to generalize knowledge learned from annotated base training data to recognize unseen novel classes has attracted considerable attention. Existing few-shot methods rely on completely clean training data. However, in the real world, the training data are always corrupted and accompanied by noise due to the disturbance in data transmission and low-quality annotation, which severely degrades the performance and generalization capability of few-shot models. To address the problem, we propose a unified peer-collaboration learning (PCL) framework to extract valid knowledge from corrupted data for few-shot learning. PCL leverages two modules to mimic the peer collaboration process which cooperatively evaluates the importance of each sample. Specifically, each module first estimates the importance weights of different samples by encoding the information provided by the other module from both global and local perspectives. Then, both modules leverage the obtained importance weights to guide the reevaluation of the loss value of each sample. In this way, the peers can mutually absorb knowledge to improve the robustness of few-shot models. Experiments verify that our framework combined with different few-shot methods can significantly improve the performance and robustness of original models.
BibTeX
@inproceedings{an2023learning,
  author = {Yuexuan An and Xingyu Zhao and Hui Xue},
  title = {{Learning to Learn from Corrupted Data for Few-Shot Learning}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2023},
  doi = {10.24963/ijcai.2023/381},
}
Figure