Abstract
SELF-ML is a novel framework designed to address the challenge of noisy or missing labels in multi-label fashion datasets. By reconstructing image features as a sparse combination of a few "cleanly" labeled basis images, the model provides an explainable mechanism to "flip" incorrect labels. This approach leverages the low-rank nature of visual feature spaces, significantly outperforming standard fine-tuning methods even when expert-verified data is extremely limited.
Introduction
In the rapidly evolving world of e-commerce, thousands of new products are uploaded daily. Manually tagging these items with multiple attributes (e.g., "V-neck," "Long Sleeved," "Formal") is both expensive and error-prone. Consequently, large-scale fashion datasets are often riddled with "noisy" labels—attributes that are either missing or incorrectly assigned.
Training Deep Convolutional Neural Networks (DCNNs) on such noisy data typically leads to poor generalization. While a small subset of data can be cleaned by experts, it is usually insufficient to train a robust model from scratch. The authors address this by proposing a method to effectively utilize massive amounts of noisy data alongside a tiny "clean" dataset to build a high-performing, explainable classifier.
Methodology
The authors propose SELF-ML, which integrates four distinct modules into a unified training pipeline:
- Feature Extractor: A standard CNN (VGG-16) that maps images into a high-dimensional feature space.
- Explainable Reconstruction Module: This core innovation assumes that the feature space of fashion images is "low-rank." The module learns a sparse vector of coefficients ($c_i$) to reconstruct an input image from a set of representative "basis" images.
- Noisy Label Flipping Module: Instead of blindly trusting original labels, this module analyzes the labels of the basis images used in the reconstruction. A learned flipping model decides whether to correct (flip) the original noisy labels based on visual evidence.
- Classifier: The final classification layer is trained using these "corrected" labels as supervision.
The system is trained jointly using a composite loss function ($\mathcal{L}_{total}$) that balances classification accuracy (focal loss), reconstruction quality (Huber loss), and label-flipping logic.
Experimental Evaluation
The researchers evaluated SELF-ML on a real-world e-commerce fashion dataset provided by Farfetch, consisting of approximately 60,200 images across four categories (dresses, jackets, knitwear, and tops). The dataset exhibited significant noise; expert-verified clean images had an average of 3.40 annotations, compared to only 1.28 in the noisy set. Only 3,000 images (~5%) were used as the clean reference set.
SELF-ML outperformed both a "noisy-only" baseline and standard fine-tuning approaches across all key metrics:
- Exact Match (0-1): SELF-ML achieved 50.00%, compared to 41.73% for fine-tuning and 1.60% for the noisy-only model.
- Jaccard Similarity: Reached 78.77%, a significant improvement over the 74.86% from fine-tuning.
- Rare Attribute Performance: The model showed dramatic gains (5% to 20% in Average Precision) on rare and visually ambiguous labels, such as specific dress silhouettes.
Discussion and Key Insights
A primary strength of SELF-ML is its explainability. Because the model reconstructs an image using only a few basis images (typically around 40 out of 1,000), the reasoning behind a label flip is transparent. Users can inspect the basis images to understand the visual evidence influencing the model's decision.
Furthermore, the approach reinforces the idea that fashion images lie on a low-dimensional manifold, where most items are variations of prototypical styles. Unlike standard transfer learning, which often overfits on small clean sets, SELF-ML uses clean data to iteratively "fix" the noisy data throughout the training process, leading to more robust feature representations.
Conclusion
While SELF-ML demonstrates impressive results on real-world data, limitations include the use of a fixed basis size and the requirement for a small, high-quality clean set. Future work will explore dynamic basis selection and evaluation on broader public benchmarks like DeepFashion to further validate the framework's scalability.
Summary updated