Abstract
Unsupervised domain adaptation (UDA) aims to train a classification model from a labeled source domain for an unlabeled target domain. Since data distributions differ across domains, models often suffer from performance degradation. Existing methods typically align global feature distributions, which can lead to misalignment by ignoring class-specific information and background noise. We propose Foreground Object Structure Transfer (FOST), a method that leverages prior knowledge to extract foreground features and performs alignment within the local structure of objects. FOST utilizes a clustering-based condition combined with relative position relationships to make same-class data distributions more compact. Experimental results on benchmarks like ImageCLEF-DA and Office-Home demonstrate that FOST achieves state-of-the-art performance.
Introduction
Deep learning models often struggle with domain shift—the discrepancy between training (source) and deployment (target) data distributions. For instance, a classifier trained on studio photos may fail on sketches. Standard UDA techniques attempt to align these domains globally, but this approach faces two major challenges:
- Background Interference: Global alignment may inadvertently align irrelevant background features, leading to "negative transfer" where performance drops.
- Structural Information Loss: High-level semantic features often discard the spatial relationships (structure) between object parts, which are crucial for robust classification across domains.
Methodology: The FOST Framework
To address these issues, we introduce Foreground Object Structure Transfer (FOST). The framework focuses on aligning foreground object structures while suppressing background noise through three key innovations.
Prior Knowledge Supervision
FOST identifies foreground objects by analyzing the gradients of ground-truth labels in the source domain. By weighting features based on their contribution to correct classification, the model focuses on "positive features" that represent the actual objects of interest, effectively creating a foreground mask without manual bounding boxes.
Structure Preservation
Unlike traditional methods that rely solely on the final, low-resolution feature maps of a backbone network, FOST merges features from earlier, shallower layers. These layers preserve the relative spatial positions of object parts (e.g., the eyes' position relative to the nose), providing a more stable structural representation for domain transfer than raw pixel values.
Structural Contrastive Loss (SCL)
In the absence of target labels, FOST employs spherical K-means clustering to assign pseudo-labels to target samples. The Structural Contrastive Loss (SCL) then minimizes intra-class distances and maximizes inter-class distances across both domains, ensuring a geometrically compact feature space.
Experimental Evaluation
We evaluated FOST on four standard UDA benchmarks: ImageCLEF-DA, Office-31, Office-Home, and VisDA-2017. FOST consistently outperformed state-of-the-art methods such as SRDC and CDAN, achieving an average accuracy of 95.3% on ImageCLEF-DA and 87.55% on the large-scale VisDA-2017 dataset.
Ablation Study
The ablation study highlights the importance of multi-layer feature integration. As shown in the results below, incorporating structural information from shallower layers ($S_2, S_3$) consistently improves the average accuracy compared to using only the deepest features ($S_4$).
Conclusion
FOST introduces a novel approach to UDA by prioritizing foreground object structures and local geometric alignment over global statistical matching. By using gradients as a mask for prior knowledge and integrating shallow structural features, FOST achieves superior generalization across diverse datasets. Future work will focus on reducing the computational overhead for large-scale cross-domain transfers and improving the robustness of foreground extraction in low-data regimes.
Summary updated