Abstract
Domain transfer techniques often rely on feature alignment to bridge the gap between source and target domains. However, standard alignment methods frequently discard task-relevant discriminative details in the pursuit of domain invariance. This paper presents the Feature Alignment and Restoration (FAR) framework, which adaptively aligns features while explicitly restoring task-critical information from the discarded residuals. By employing a dual ranking entropy loss, FAR ensures that restored features enhance classification confidence. Experimental results across multiple benchmarks demonstrate that FAR achieves state-of-the-art performance in both unsupervised domain adaptation and domain generalization.
Introduction
In machine learning, "domain shift" occurs when a model trained on one type of data (e.g., professional photos) fails on another (e.g., hand-drawn sketches). To mitigate this, researchers typically employ Feature Alignment (FA), which forces the model to learn domain-invariant representations. However, alignment is often "task-ignorant." In the process of making features from different domains match, the model may discard discriminative details that are domain-specific yet crucial for the task.
As illustrated in Figure 1, if a model relies solely on "shape" to align photos and sketches, it might lose the "texture" information necessary to distinguish a horse from a sheep in the photo domain. The FAR framework is designed to overcome this trade-off by recovering these lost signals.
Methodology: The FAR Framework
The proposed FAR framework addresses the limitations of standard alignment through a two-phase process involving adaptive alignment and information restoration.
1. Feature Alignment (FA)
Rather than aligning all features indiscriminately, FAR utilizes spatial and channel attention to adaptively select a subspace of features most suitable for alignment. It then matches the moments (mean and variance) of these distributions across domains. This selective approach preserves more raw information for the subsequent restoration phase compared to global alignment strategies.
2. Feature Restoration (FR)
This core component calculates a "residual" — the difference between the original features and the aligned ones. An attention module distills task-relevant information from this residual, which is then added back to the aligned features to compensate for lost discriminative power. This ensures the final representation is both domain-invariant and task-optimized.
Dual Ranking Entropy Loss
To ensure the restoration step captures useful information rather than domain-specific noise, the authors introduce a Dual Ranking Entropy Loss. This loss encourages the enhanced feature (aligned + task-relevant residual) to yield lower entropy, indicating higher prediction confidence. Conversely, it forces a contaminated feature (aligned + task-irrelevant residual) to yield higher entropy, indicating lower confidence. This mechanism supervises the disentanglement of relevant features without requiring explicit labels for what constitutes "relevance."
Experimental Evaluation
The FAR framework was evaluated on four major benchmarks: Digit-Five, mini-DomainNet, PACS, and Office-Home.
Key Results
FAR consistently outperformed existing state-of-the-art (SOTA) methods across both adaptation and generalization tasks:
- Unsupervised Domain Adaptation (UDA): On the Digit-Five benchmark, FAR achieved an average accuracy of 90.39%, outperforming the previous SOTA (M3SDA-β) by 2.74%. On mini-DomainNet, it reached 57.03%, a 1.51% improvement over the next best method.
- Domain Generalization (DG): FAR achieved top results on PACS (81.7%) and Office-Home (66.0%), demonstrating robust generalization to completely unseen domains.
Qualitative analysis via t-SNE visualizations indicates that FAR produces cleaner, more separable class clusters compared to standard alignment methods, which often cause class overlap in the shared feature space as they force alignment at the expense of class boundaries.
Conclusion
The FAR framework demonstrates that information discarded during alignment is not merely noise but often contains task-critical details. By treating the alignment residual as a source for restoration and using entropy ranking for disentanglement, FAR effectively bridges the domain gap without sacrificing discriminative accuracy. Future work will explore extending this framework to dense prediction tasks such as image segmentation and object detection.
Summary updated