Improved Knowledge Transfer for Semi-supervised Domain Adaptation via Trico Training Strategy

Ba Hung Ngo, Yeon Jeong Chae, Jung Eun Kwon, Jae Hyeon Park, Sung In Cho

This ICCV 2023 paper introduces a novel framework to help AI models generalize better when they have plenty of data from one "source" domain but very little labeled data in a "target" domain. The proposed method, TriCT (Trico-training), leverages three specialized classifiers that "teach" each other through a mutual exchange of pseudo-labels and a novel "pseudo-edge" concept.

The Challenge: Domain Discrepancies

In Semi-supervised Domain Adaptation (SSDA), the goal is to train a model on a massive labeled dataset (Source) and adapt it to a target domain where labeled data is scarce but unlabeled data is plentiful. For example, adapting a model trained on photos to recognize objects in sketches.

Illustration of inter-domain and intra-domain discrepancies in SSDA.

As shown in Figure 1, the two main hurdles are:

  1. Inter-domain discrepancy: The fundamental difference between domains (e.g., photos vs. sketches), known as "domain shift."
  2. Intra-domain discrepancy: The gap within the target domain itself, where the model might only align unlabeled samples that look very similar to the few labeled ones, leaving the rest misaligned.

Methodology: Trico-training (TriCT)

The core of the paper is the Trico-training (TriCT) strategy, which employs a shared feature extractor and three distinct classifiers to capture different aspects of the data structure.

The TriCT framework, including data preparation, training procedure, and the triple-classifier interaction strategy.

The Triple-Threat Architecture

  • MLP Classifier ($F_{mlp}$): Focuses on the semantic information of individual images.
  • Inter-view GCN ($G_{gcn}^{inter}$): Trained primarily on source data to provide a "source-eye view" of the target data.
  • Intra-view GCN ($G_{gcn}^{intra}$): Trained on the limited labeled target data to capture the specific structure of the target domain.

Co-training Interactions

The "Trico" part comes from three clever co-training interactions:

  1. MLP $\leftrightarrow$ Inter-view GCN: The GCN provides pseudo-labels to the MLP to help it align across domains. In return, the MLP provides "pseudo-edges"—it tells the GCN which unlabeled images likely belong to the same class, allowing the GCN to build a more accurate relationship graph.
  2. MLP $\leftrightarrow$ Intra-view GCN: A similar exchange happens here to reduce the gap between labeled and unlabeled target data.
  3. Inter-view $\leftrightarrow$ Intra-view GCN: Because one GCN is "source-heavy" and the other is "target-heavy," they exchange labels to balance out their respective biases.

Key Insights

  • The Power of Pseudo-Edges: Instead of just guessing labels, the MLP helps the GCNs "see" the connections between unlabeled samples. By defining these edges, the GCN can aggregate features from similar neighbors, making its own predictions much more robust.
  • GCNs as Auxiliary Teachers: While MLPs are great at individual features, GCNs are superior at mining data structures. Using GCNs to "teach" the MLP allows the final model to benefit from relational data it would otherwise ignore.

Experimental Results

The researchers evaluated TriCT on three standard SSDA benchmarks: Office-31, Office-Home, and the massive DomainNet dataset. They tested these in 1-shot and 3-shot settings (where only 1 or 3 labeled examples per class were available in the target domain).

TriCT consistently surpassed previous top-tier methods:

  • DomainNet: On the ResNet-34 backbone, TriCT reached 85.5% accuracy in the 3-shot setting, beating the previous best (DEEM) by 6.0%.
  • Office-Home: It achieved 86.7%, a massive jump over the previous state-of-the-art (MCL at 77.1%).
  • Visual Proof: t-SNE visualizations demonstrated that TriCT creates much tighter, cleaner clusters of target data and aligns them more closely with source data than baseline models.

Conclusion

TriCT effectively closes the gap between different data domains by combining the semantic strengths of MLPs with the relational strengths of GCNs. While the training process involves three classifiers, the final inference only requires the MLP, making it an efficient solution for real-world domain adaptation tasks.

Summary updated

Abstract

The motivation of the semi-supervised domain adaptation (SSDA) is to train a model by leveraging knowledge acquired from the plentiful labeled source combined with extremely scarce labeled target data to achieve the lowest error on the unlabeled target data at the testing time. However, due to inter-domain and intra-domain discrepancies, the improvement of classification accuracy is limited. To solve these, we propose the Trico-training method that utilizes a multilayer perceptron (MLP) classifier and two graph convolutional network (GCN) classifiers called interview GCN and intra-view GCN classifiers. The first co-training strategy exploits a correlation between MLP and inter-view GCN classifiers to minimize the inter-domain discrepancy, in which the inter-view GCN classifier provides its pseudo labels to teach the MLP classifier, which encourages class representation alignment across domains. In contrast, the MLP classifier gives feedback to the inter-view GCN classifier by using a new concept, ‘pseudo-edge’, for neighbor’s feature aggregation. Doing this increases the data structure mining ability of the inter-view GCN classifier; thus, the quality of generated pseudo labels is improved. The second co-training strategy between MLP and intra-view GCN is conducted in a similar way to reduce the intra-domain discrepancy by enhancing the correlation between labeled and unlabeled target data. Due to an imbalance in classification accuracy between inter-view and intra-view GCN classifiers, we propose the third co-training strategy that encourages them to cooperate to address this problem. We verify the effectiveness of the proposed method on three standard SSDA benchmark datasets: Office-31, Office-Home, and DomainNet. The extended experimental results show that our method surpasses the prior state-of-the-art approaches in SSDA.
BibTeX
@inproceedings{ngo2023improved,
  author = {Ba Hung Ngo and Yeon Jeong Chae and Jung Eun Kwon and Jae Hyeon Park and Sung In Cho},
  title = {{Improved Knowledge Transfer for Semi-supervised Domain Adaptation via Trico Training Strategy}},
  booktitle = {IEEE/CVF International Conference on Computer Vision},
  year = {2023},
  doi = {10.1109/iccv51070.2023.01760},
}
Figure