Abstract
Deep Neural Networks (DNNs) are highly susceptible to noisy labels, often memorizing incorrect associations which hinders generalization. MetaCleaner is a plug-and-play module that addresses this by learning to "hallucinate" clean representations from noisy subsets. By analyzing the relationships within a small group of images from the same category, MetaCleaner identifies outliers and generates a synthetic, clean feature vector. Evaluated on large-scale benchmarks like Food-101N and Clothing1M, MetaCleaner achieves state-of-the-art performance, demonstrating robustness to significant label noise without the need for expensive manual verification.
Introduction: The Challenge of Noisy Labels
The success of deep learning is largely driven by the availability of massive, high-quality datasets. However, obtaining clean, manually labeled data at scale is prohibitively expensive. A common alternative is to scrape images from the web using search keywords. While efficient, this method introduces "noisy labels"—images that are incorrectly categorized due to the limitations of search engines or the inherent ambiguity of web content. For instance, a search for "steak" might return images of hamburgers or pizzas.
Standard DNNs are powerful learners but lack the inherent ability to distinguish between correct and incorrect labels during training. If a model is repeatedly told a pizza is a steak, it will eventually memorize this error, degrading its performance on actual steak images. Previous attempts to solve this involved label correction or individual sample weighting. However, these methods often ignore the broader context: an image's "noisiness" is best determined by comparing it to other images within the same purported category.
Methodology: Hallucinating Clean Features
MetaCleaner is designed as a flexible module that can be integrated between a feature extractor (e.g., ResNet) and a final classifier. Its core innovation lies in treating noise detection as a relative task rather than an absolute one.
1. Noisy Weighting
Instead of processing images in isolation, MetaCleaner takes a small subset of $N$ images (typically $N=4$) from a single category. The features extracted by a CNN are concatenated and passed through a multi-layer perceptron (MLP). This allows the model to learn the consensus of the group. If the majority of images share similar visual features (e.g., grilled meat for "steak"), the MLP assigns them high confidence scores ($\alpha$). Conversely, outliers like a pizza in a steak subset receive low scores.
2. Clean Hallucinating
The module then computes a weighted average of the subset's features based on these confidence scores. This process "hallucinates" a single clean representation ($v_c$) that captures the essential features of the category while suppressing the influence of noisy samples. This synthetic feature vector serves as a more stable and accurate target for the classifier during training.
Importantly, MetaCleaner is only active during the training phase. Once the model has learned robust features, the module can be removed, allowing the classifier to operate on individual images during inference.
Experimental Evaluation
The effectiveness of MetaCleaner was tested on several challenging benchmarks:
- Food-101N: A dataset of 310,000 food images with approximately 20% noise. MetaCleaner achieved 85.05% accuracy, surpassing previous methods like CleanNet without requiring its specialized verification labels.
- Clothing1M: A massive dataset of 1 million clothing images with roughly 40% noise. MetaCleaner reached 72.50% accuracy using only noisy data, and 80.78% when fine-tuned with a small clean subset.
- Synthetic Noise (CIFAR-10 & ImageNet): To test the limits of the approach, researchers injected 40% synthetic noise. On CIFAR-10, MetaCleaner improved accuracy from a 67.97% baseline to 90.09%.
Discussion and Key Insights
The success of MetaCleaner highlights several critical points in noisy-label learning:
- Contextual Awareness: By processing subsets, the model utilizes the "majority rule" to identify noise, which is far more effective than analyzing images in a vacuum.
- Feature Blending as Augmentation: The "hallucination" process acts as a form of vicinal risk minimization. By blending features, the model is exposed to a neighborhood of data points, making it more robust to individual variations and errors.
- Ease of Integration: The module's simplicity—consisting of a few fully connected layers—makes it a practical "plug-and-play" solution for various architectures and tasks.
Limitations and Future Directions
Despite its performance, MetaCleaner has some constraints. The choice of subset size ($N$) is a critical hyperparameter; too small a subset fails to establish a consensus, while too large a subset increases computational costs. Furthermore, in scenarios where noise dominates a category (i.e., the "outliers" are the majority), the model may struggle to identify the correct representation. Future work may involve exploring theoretical foundations or adapting the module for extreme noise conditions.
Summary updated