Abstract
Source-free domain adaptation (SFDA) aims to adapt a pre-trained source model to a target domain without access to the original training data. This paper introduces Neighborhood Reciprocity Clustering (NRC), a method that leverages the intrinsic local structure of target data to guide adaptation. By identifying "reciprocal" nearest neighbors—pairs that mutually rank each other as top neighbors—the method builds highly reliable clusters. An extension, NRC++, incorporates density-based outlier detection to further refine the adaptation process. NRC achieves state-of-the-art performance across diverse 2D and 3D benchmarks, demonstrating that mutual friendship in feature space is a powerful proxy for semantic consistency.
1. Introduction
Deep learning models often suffer from performance degradation when deployed on data from a different distribution than their training set, a phenomenon known as domain shift. While Unsupervised Domain Adaptation (UDA) has traditionally addressed this by aligning source and target distributions, it typically requires simultaneous access to both datasets. In many real-world scenarios, such as medical imaging or proprietary industrial datasets, the source data is unavailable due to privacy regulations or intellectual property concerns.
Source-Free Domain Adaptation (SFDA) addresses this constraint by adapting a model using only the pre-trained source weights and unlabeled target data. The challenge lies in guiding the adaptation without the ground-truth anchors provided by the source distribution.
2. Methodology: Neighborhood Reciprocity Clustering (NRC)
The core philosophy of NRC is that even when global distribution shift occurs, the local neighborhood structure of the target data remains informative. The authors propose that samples belonging to the same class will naturally cluster together in the feature space, provided we can identify which neighbors are truly reliable.
2.1 Reciprocal Nearest Neighbors (RNN)
Standard nearest neighbor approaches are often noisy, as a point might have a neighbor from a different class that happens to be nearby due to domain shift. NRC introduces "reciprocity" as a filter for reliability. If sample A is a nearest neighbor of B, and B is also a nearest neighbor of A, they are considered "reciprocal neighbors" or "mutual friends." The algorithm assigns higher affinity weights to these reciprocal pairs, forcing the model to produce consistent predictions for them while assigning lower weights to non-reciprocal neighbors.
2.2 Expanded Neighborhoods and Manifold Walking
To capture broader context, NRC looks beyond immediate neighbors to "neighbors of neighbors." This strategy effectively "walks the manifold" of the data. Interestingly, the authors found that when a sample appears multiple times within these expanded neighborhoods (e.g., as a neighbor to multiple primary neighbors), treating these duplicates as a signal for increased affinity improves performance. This naturally weights the most central and consistent points in a cluster more heavily.
2.3 NRC++: Outlier Detection
Not all target samples are equally useful for adaptation. Some may lie in low-density regions of the feature space, acting as outliers that can poison cluster formation if forced into a group. NRC++ introduces a density-based filtering mechanism. By counting how frequently a sample appears as a neighbor to others, the model identifies and excludes low-density outliers from the clustering loss, ensuring the adaptation is driven by the most representative data points.
3. Experimental Evaluation
The authors evaluated NRC across a wide array of benchmarks, spanning both 2D images and 3D point clouds:
- 2D Image Recognition: Testing on Office-31, Office-Home, and the large-scale VisDA dataset (synthetic-to-real).
- 3D Point Cloud Recognition: Evaluation on PointDA-10, which includes ModelNet, ShapeNet, and ScanNet.
- Massive Scale: Testing on DomainNet, featuring six distinct domains and significant complexity.
4. Key Insights
The success of NRC is rooted in several empirical observations:
- Trustworthiness of Reciprocity: The authors show that reciprocal neighbors share the same ground-truth label approximately 75% of the time, a significantly higher rate than standard one-way neighbors.
- Manifold vs. Radius: Expanding the neighborhood by looking at "neighbors of neighbors" is more effective than simply increasing the search radius. The former follows the geometric structure of the data, while the latter risks including samples from different semantic clusters.
- The Importance of Filtering: The transition from NRC to NRC++ demonstrates that in unsupervised learning, the ability to ignore noisy or isolated data points is as critical as the ability to cluster related ones.
5. Results and Conclusion
NRC++ sets new benchmarks for source-free adaptation. On the VisDA dataset, it achieved 88.1% accuracy, significantly outperforming previous SFDA leaders like SHOT (82.9%) and even surpassing some methods that utilize source data (e.g., RWOT at 84.0%). In the 3D domain, NRC++ reached 55.1% on PointDA-10, outperforming specialized 3D methods like PointDAN.
While the method requires careful hyperparameter selection (such as the number of neighbors $K$) and a memory bank for feature storage, its ability to perform multi-source and multi-target adaptation without architectural changes makes it a robust solution for modern, privacy-conscious machine learning pipelines.
Summary updated