MHPL: Minimum Happy Points Learning for Active Source Free Domain Adaptation

Fan Wang, Zhongyi Han, Zhiyan Zhang, Rundong He, Yilong Yin

Abstract

Source-Free Domain Adaptation (SFDA) addresses the challenge of adapting a pre-trained source model to a target domain without access to original source data, a common requirement in privacy-sensitive applications. Active SFDA (ASFDA) further enhances this by allowing a small fraction of target data to be labeled. However, existing active learning methods often struggle with model miscalibration on the target domain. We propose Minimum Happy Points Learning (MHPL), a novel framework that identifies the most informative "Minimum Happy" (MH) points—samples that are source-dissimilar and reside in chaotic neighborhood environments. By leveraging local neighbor structures rather than unreliable model confidence, MHPL achieves significant performance gains across multiple benchmarks, including a 6.5% improvement on the Office-Home dataset with only a 5% labeling budget.

Introduction

In many real-world scenarios, such as healthcare or security, data privacy prevents the sharing of raw source datasets. Source-Free Domain Adaptation (SFDA) has emerged to handle these constraints, but it often faces a performance bottleneck due to the lack of direct domain discrepancy measurements. Active SFDA (ASFDA) attempts to overcome this by incorporating a human-in-the-loop to label a tiny portion (e.g., 5%) of the target data.

Standard active learning strategies typically rely on uncertainty metrics like entropy or least confidence. In the context of domain shift, these metrics are often unreliable because the source model is miscalibrated on the target domain, leading to overconfident but incorrect predictions. MHPL addresses this by focusing on the "unhappiest" points—those that the model finds most difficult based on their local feature-space environment.

Methodology: Identifying Minimum Happy Points

The core of MHPL is the selection of Minimum Happy (MH) points. These are samples that are neighbor-chaotic, diverse, and source-dissimilar. The framework consists of four primary components:

  1. Neighbor Environment Uncertainty (NEU): Instead of relying on the model's self-prediction, NEU evaluates a sample's neighborhood. It combines Neighbor Purity (label consistency among neighbors) and Neighbor Affinity (spatial proximity) to identify samples in chaotic regions where the model is likely struggling.
  2. Neighbor Diversity Relaxation (NDR): To ensure the selected active set covers the target distribution effectively, NDR prevents the selection of multiple samples from the same local neighborhood.
  3. One-shot Querying: MHPL utilizes the raw source model for a single round of querying. The authors find that the initial, un-adapted model serves as the best "yardstick" for identifying samples that are truly dissimilar to the source domain.
  4. Neighbor Focal (NF) Loss: During the adaptation phase, this loss function prioritizes learning from the labeled MH points, weighting them by their neighbor purity to ensure robust generalization.
Figure 2: t-SNE visualization comparing sampling strategies. MHPL (d) selects points (red stars) that are more representative of the underlying data structure compared to standard active learning baselines like BVSB (b) and CoreSet (c).

Experimental Results

The MHPL framework was evaluated on three major domain adaptation benchmarks: Office-Home, VisDA-2017, and Office-31.

Performance Gains

MHPL consistently outperformed both state-of-the-art SFDA methods and established Active Learning strategies. On the Office-Home dataset, MHPL achieved an average accuracy of 79.3%, which is a 6.5% improvement over the SFDA method A2Net and a 2.3% lead over the previous best ASFDA method, ELPT.

Figure 1: Classification accuracy comparison on the Office-Home dataset. MHPL significantly outperforms baseline SFDA (SHOT) and various active learning combinations.

Label Efficiency

A key finding of the study is the high label efficiency of MHPL. With only a 5% labeling budget, the method effectively bridges the gap between source-free adaptation and methods that require full source data access. This demonstrates that identifying "Minimum Happy" points is a superior strategy for selecting informative samples in the presence of domain shift.

Conclusion

MHPL introduces a robust approach to Active Source-Free Domain Adaptation by shifting the focus from model-centric uncertainty to data-centric neighbor structures. By identifying "Minimum Happy" points that are both source-dissimilar and neighborhood-chaotic, the framework overcomes the common pitfall of model miscalibration. Future work may involve optimizing the computational cost of neighbor searches for large-scale datasets and further exploring the theoretical properties of source-dissimilarity in active learning.

Summary updated

Abstract

Source free domain adaptation (SFDA) aims to transfer a trained source model to the unlabeled target domain without accessing the source data. However, the SFDA setting faces a performance bottleneck due to the absence of source data and target supervised information, as evidenced by the limited performance gains of the newest SFDA methods. Active source free domain adaptation (ASFDA) can break through the problem by exploring and exploiting a small set of informative samples via active learning. In this paper, we first find that those satisfying the proper-ties of neighbor-chaotic, individual-different, and source-dissimilar are the best points to select. We define them as the minimum happy (MH) points challenging to explore with existing methods. We propose minimum happy points learning (MHPL) to explore and exploit MH points actively. We design three unique strategies: neighbor environment uncertainty, neighbor diversity relaxation, and one-shot querying, to explore the MH points. Further, to fully exploit MH points in the learning process, we design a neighbor focal loss that assigns the weighted neighbor purity to the cross entropy loss of MH points to make the model focus more on them. Extensive experiments verify that MHPL remarkably exceeds the various types of baselines and achieves significant performance gains at a small cost of labeling.
BibTeX
@inproceedings{wang2023mhpl,
  author = {Fan Wang and Zhongyi Han and Zhiyan Zhang and Rundong He and Yilong Yin},
  title = {{MHPL: Minimum Happy Points Learning for Active Source Free Domain Adaptation}},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year = {2023},
  doi = {10.1109/cvpr52729.2023.01916},
}
Figure