This ICCV 2023 paper introduces a novel framework to help AI models generalize better when they have plenty of data from one "source" domain but very little labeled data in a "target" domain. The proposed method, TriCT (Trico-training), leverages three specialized classifiers that "teach" each other through a mutual exchange of pseudo-labels and a novel "pseudo-edge" concept.
The Challenge: Domain Discrepancies
In Semi-supervised Domain Adaptation (SSDA), the goal is to train a model on a massive labeled dataset (Source) and adapt it to a target domain where labeled data is scarce but unlabeled data is plentiful. For example, adapting a model trained on photos to recognize objects in sketches.
As shown in Figure 1, the two main hurdles are:
- Inter-domain discrepancy: The fundamental difference between domains (e.g., photos vs. sketches), known as "domain shift."
- Intra-domain discrepancy: The gap within the target domain itself, where the model might only align unlabeled samples that look very similar to the few labeled ones, leaving the rest misaligned.
Methodology: Trico-training (TriCT)
The core of the paper is the Trico-training (TriCT) strategy, which employs a shared feature extractor and three distinct classifiers to capture different aspects of the data structure.
The Triple-Threat Architecture
- MLP Classifier ($F_{mlp}$): Focuses on the semantic information of individual images.
- Inter-view GCN ($G_{gcn}^{inter}$): Trained primarily on source data to provide a "source-eye view" of the target data.
- Intra-view GCN ($G_{gcn}^{intra}$): Trained on the limited labeled target data to capture the specific structure of the target domain.
Co-training Interactions
The "Trico" part comes from three clever co-training interactions:
- MLP $\leftrightarrow$ Inter-view GCN: The GCN provides pseudo-labels to the MLP to help it align across domains. In return, the MLP provides "pseudo-edges"—it tells the GCN which unlabeled images likely belong to the same class, allowing the GCN to build a more accurate relationship graph.
- MLP $\leftrightarrow$ Intra-view GCN: A similar exchange happens here to reduce the gap between labeled and unlabeled target data.
- Inter-view $\leftrightarrow$ Intra-view GCN: Because one GCN is "source-heavy" and the other is "target-heavy," they exchange labels to balance out their respective biases.
Key Insights
- The Power of Pseudo-Edges: Instead of just guessing labels, the MLP helps the GCNs "see" the connections between unlabeled samples. By defining these edges, the GCN can aggregate features from similar neighbors, making its own predictions much more robust.
- GCNs as Auxiliary Teachers: While MLPs are great at individual features, GCNs are superior at mining data structures. Using GCNs to "teach" the MLP allows the final model to benefit from relational data it would otherwise ignore.
Experimental Results
The researchers evaluated TriCT on three standard SSDA benchmarks: Office-31, Office-Home, and the massive DomainNet dataset. They tested these in 1-shot and 3-shot settings (where only 1 or 3 labeled examples per class were available in the target domain).
TriCT consistently surpassed previous top-tier methods:
- DomainNet: On the ResNet-34 backbone, TriCT reached 85.5% accuracy in the 3-shot setting, beating the previous best (DEEM) by 6.0%.
- Office-Home: It achieved 86.7%, a massive jump over the previous state-of-the-art (MCL at 77.1%).
- Visual Proof: t-SNE visualizations demonstrated that TriCT creates much tighter, cleaner clusters of target data and aligns them more closely with source data than baseline models.
Conclusion
TriCT effectively closes the gap between different data domains by combining the semantic strengths of MLPs with the relational strengths of GCNs. While the training process involves three classifiers, the final inference only requires the MLP, making it an efficient solution for real-world domain adaptation tasks.
Summary updated