Abstract
Domain adaptation (DA) becomes an up-and-coming technique to address the insufficient or no annotation issue by exploiting external source knowledge. Existing DA algorithms mainly focus on practical knowledge transfer through domain alignment. Unfortunately, they ignore the fairness issue when the auxiliary source is extremely imbalanced across different categories, which results in severe under-presented knowledge adaptation of minority source set. To this end, we propose a Towards Fair Knowledge Transfer (TFKT) framework to handle the fairness challenge in imbalanced cross-domain learning. Specifically, a novel cross-domain knowledge propagation technique is proposed with the guidance of within-source and cross-domain structure graphs to smooth the manifold of the minority source set. Besides, a cross-domain fulfillment augmentation strategy is exploited achieve domain adaptation. Moreover, hybrid distinct classifiers and cross-domain prototype alignment are adopted to seek a more robust classifier boundary and mitigate the domain shift. Such three strategies are formulated into a unified framework to address the fairness issue and domain shift challenge. Extensive experiments over two popular benchmarks have verified the effectiveness of our proposed model by comparing to existing state-of-the-art DA models, and especially our model significantly improves over 20% on two benchmarks in terms of the overall accuracy.
Introduction
Domain Adaptation (DA) is a cornerstone of modern machine learning, enabling the transfer of knowledge from a labeled source domain to an unlabeled target domain. However, a significant oversight in existing DA literature is the assumption of a balanced source domain. In real-world scenarios, data often follows a "long-tailed" distribution where a few majority classes dominate, and many minority classes have very few samples.
When standard DA models encounter such imbalance, they naturally prioritize majority classes, leading to a "transfer fairness" issue. Minority classes are effectively ignored during the alignment process, resulting in poor performance on those categories in the target domain. The TFKT framework is designed to address this extreme imbalance, even in cases where minority classes have as few as one labeled sample (1-shot learning).
Methodology
The Towards Fair Knowledge Transfer (TFKT) framework addresses the fairness gap through three integrated components:
1. Cross-Domain Feature Augmentation (CDA)
To combat the noise inherent in few-shot samples, TFKT utilizes Knowledge Propagation. By constructing structure graphs that capture relationships both within the source and across domains, the model "smooths" the feature space. This process acts as a denoiser, moving minority samples toward their true class manifolds. Following this, Fulfillment Augmentation generates synthetic features by interpolating refined source embeddings with cross-domain information, effectively bridging the distribution gap between domains.
2. Hybrid Distinct Classifiers
Recognizing that majority and minority classes require different decision logic, TFKT employs a dual-classifier system. A standard neural network classifier ($C_N$) is optimized for majority classes with abundant data. Conversely, a prototypical classifier ($C_P$) handles minority classes. Prototypical classifiers are inherently more robust for few-shot learning as they rely on distance-to-center metrics rather than complex, data-hungry decision boundaries.
3. Cross-Domain Prototype Alignment (CPA)
To mitigate domain shift, the model aligns the source and target domains by minimizing the distance between their respective class prototypes (centroids). This ensures that the categorical knowledge remains consistent across the domain boundary.
Experimental Results
The researchers evaluated TFKT on the Office-31 and Office-Home benchmarks. To simulate extreme imbalance, they restricted the first 10 (Office-31) or 25 (Office-Home) categories to only 1 or 5 labeled samples.
As shown in the results for Office-Home (5-shot), TFKT significantly outperforms state-of-the-art models like DAN, MCD, and SymNets. The most notable improvement is in the minority class accuracy ($A_f$), where TFKT often achieves gains of over 30% compared to traditional methods. On average, TFKT reached 65.6% overall accuracy ($A_o$) on Office-Home, a substantial leap from the 51.1% achieved by DAN. Even in the challenging 1-shot scenario, TFKT maintained stable performance while other models' accuracies plummeted.
Conclusion
The TFKT framework demonstrates that explicitly addressing class imbalance is vital for fair and effective domain adaptation. By combining graph-based smoothing, cross-domain augmentation, and specialized classifiers, TFKT provides a robust solution for real-world, long-tailed data scenarios. Future work may involve optimizing the computational cost of graph construction for larger datasets and exploring image-level generative augmentation strategies.
Summary updated