Semi-supervised Learning with a Teacher-student Network for Generalized Attribute Prediction

Minchul Shin

Abstract

This paper presents a study on semi-supervised learning to solve the visual attribute prediction problem. In many applications of vision algorithms, the precise recognition of visual attributes of objects is important but still challenging. This is because defining a class hierarchy of attributes is ambiguous, so training data inevitably suffer from class imbalance and label sparsity, leading to a lack of effective annotations. An intuitive solution is to find a method to effectively learn image representations by utilizing unlabeled images. With that in mind, we propose a multi-teacher-single-student (MTSS) approach inspired by the multi-task learning and the distillation of semi-supervised learning. Our MTSS learns task-specific domain experts called teacher networks using the label embedding technique and learns a unified model called a student network by forcing a model to mimic the distributions learned by domain experts. Our experiments demonstrate that our method not only achieves competitive performance on various benchmarks for fashion attribute prediction, but also improves robustness and cross-domain adaptability for unseen domains.

Introduction

Visual Attribute Prediction (VAP) involves identifying specific qualities of an object, such as the color, pattern, or cut of a garment. Despite its utility in e-commerce and image retrieval, VAP remains a significant challenge for machine learning due to several factors:

  1. Subjectivity and Ambiguity: Attribute definitions are often subjective (e.g., distinguishing between "navy" and "black"), making strict hierarchical labeling difficult.
  2. The Long Tail: Data distributions are heavily skewed; common attributes like "black" are overrepresented, while rare patterns like "leopard print" suffer from label sparsity.
  3. Annotation Expense: High-quality, multi-attribute labeling is labor-intensive and costly.

While existing solutions often rely on auxiliary data like human-annotated landmarks, this research proposes a domain-agnostic solution that leverages unlabeled data through a Multi-Teacher-Single-Student (MTSS) framework.

Methodology

The proposed MTSS approach is a two-stage semi-supervised learning framework designed to create a robust, unified model from specialized experts.

Stage 1: Training the Experts (Teachers)

Instead of training a single monolithic model, the researchers train several "Teacher" networks, each specialized in a specific attribute domain (e.g., texture, fabric, or shape). To handle label sparsity and the relationships between attributes, they utilize Label Embedding (LE).

Overview of the Label Embedding (LE) technique used to map images and attributes into a shared latent space.

As shown in the figure above, LE maps both images and text labels into a shared latent space. This allows the model to recognize semantic similarities—understanding that "polka dot" and "striped" are both patterns—even when visual features differ. A modified focal loss is also applied to ensure teachers focus on "hard" examples during training.

Stage 2: Knowledge Distillation (The Student)

A single "Student" network is then trained to mimic the output distributions of all teacher networks simultaneously. The core innovation lies in using unlabeled data during this phase. By forcing the student to match the teachers' feature distributions on images the teachers have not seen, the student learns a more generalized and robust representation than any individual teacher.

Comparison of learning paradigms: (a) Common, (b) Progressive, and (c) the proposed MTSS framework.

Unlike common multi-task learning, the MTSS framework (illustrated in part (c) above) uses distillation to unify knowledge, preventing the student from overfitting to the specific biases of a small labeled training set.

Experimental Results

The model was evaluated on four major fashion datasets: DeepFashion, iMaterialist-Fashion-2018, DARN, and FiccY (a real-world production dataset).

Key Findings

  • Superior Performance: On the DeepFashion benchmark, the student model achieved a Recall@3 of 58.02% and a Recall@5 of 64.35%, outperforming several models that utilized extra auxiliary information like pose data.
  • Cross-Domain Adaptability: When trained on professional catalog photos (DeepFashion) and tested on user-uploaded street photos (DARN), the student model showed significantly better generalization than its individual teachers.
  • Robustness to Noise: The student model demonstrated higher stability against 15 types of image corruption (e.g., blur, digital noise). By aggregating knowledge from multiple experts, the student effectively filters out outlier predictions caused by corrupted input.

Conclusion

The MTSS framework provides a powerful, domain-agnostic method for visual attribute prediction. By combining label embedding with semi-supervised distillation, the approach overcomes the limitations of sparse and imbalanced data. While the student's performance is currently bounded by the quality of its teachers, future work will focus on identifying and suppressing noisy signals from teacher networks to further enhance the student's accuracy and robustness.

Summary updated

Abstract

This paper presents a study on semi-supervised learning to solve the visual attribute prediction problem. In many applications of vision algorithms, the precise recognition of visual attributes of objects is important but still challenging. This is because defining a class hierarchy of attributes is ambiguous, so training data inevitably suffer from class imbalance and label sparsity, leading to a lack of effective annotations. An intuitive solution is to find a method to effectively learn image representations by utilizing unlabeled images. With that in mind, we propose a multi-teacher-single-student (MTSS) approach inspired by the multi-task learning and the distillation of semi-supervised learning. Our MTSS learns task-specific domain experts called teacher networks using the label embedding technique and learns a unified model called a student network by forcing a model to mimic the distributions learned by domain experts. Our experiments demonstrate that our method not only achieves competitive performance on various benchmarks for fashion attribute prediction, but also improves robustness and cross-domain adaptability for unseen domains.
BibTeX
@inproceedings{shin2020semi,
  author = {Minchul Shin},
  title = {{Semi-supervised Learning with a Teacher-student Network for Generalized Attribute Prediction}},
  booktitle = {European Conference on Computer Vision},
  year = {2020},
  doi = {10.1007/978-3-030-58621-8\_30},
}
Figure