Introduction
In 3D computer vision, models often face a "domain gap" when transitioning from synthetic training data (e.g., CAD models) to real-world lidar scans. Unsupervised Domain Adaptation (UDA) aims to bridge this gap without target-domain labels. A common strategy involves Multi-Task Learning (MTL), where a model performs a primary classification task alongside self-supervised auxiliary tasks. However, these tasks can produce conflicting gradients, leading to "negative transfer" where auxiliary learning actually degrades performance on specific classes. This paper introduces a novel method to identify and mitigate these conflicts by analyzing the statistical properties of model attention.
Methodology: The SM-DSB Framework
The researchers propose the Saliency Map-based Data Sampling Block (SM-DSB), a plug-and-play module designed to filter out detrimental training samples. The framework operates through two primary components: the Measurer and the Selector.
- The Measurer: For each input point cloud, the Measurer generates a saliency map—a heatmap indicating the importance of specific points to the model's prediction. It then calculates the skewness (a 3rd-order statistic) of this map. High skewness indicates the model is hyper-focused on a few points (shortcut learning), while low skewness suggests a more robust, distributed focus.
- The Selector: Using an adaptive threshold based on the calculated skewness, the Selector identifies samples likely to cause gradient conflicts. It then "filters" these samples, preventing their self-supervised gradients from interfering with the model update during that training batch.
The Skewness-Conflict Correlation
A central contribution of this work is the discovery of a strong correlation between saliency map skewness and gradient conflict. When a model's attention is highly skewed, it becomes vulnerable to domain shifts—for instance, if a real-world sensor misses the specific points the model has over-indexed on.
As shown in the correlation analysis, there is a clear negative relationship (Correlation Coefficient = -0.63) between skewness and the cosine similarity gain of gradients relative to an "oracle" gradient (the ideal gradient path if labels were available). By targeting high-skewness samples, the SM-DSB effectively performs "gradient surgery" to keep the training trajectory aligned with the optimal path.
Experimental Results
The SM-DSB was evaluated on two major benchmarks: PointDA-10 (classification) and PointSegDA (segmentation).
- State-of-the-Art Performance: Integrated into the "Self-dist-GCN" framework, SM-DSB achieved top results in 5 out of 6 PointDA-10 tasks. Notably, it reached 90.5% accuracy on the challenging Real-to-Synthetic (ScanNet to ModelNet) task.
- Class-Level Gains: The method significantly improved accuracy for difficult classes. For example, accuracy for the "plant" class increased by nearly 20% compared to baselines that did not account for gradient interference.
- Segmentation Robustness: On the PointSegDA benchmark, the method improved the mean Intersection over Union (mIoU) in 10 out of 12 transfer directions, demonstrating its effectiveness across different 3D tasks.
Conclusion
This research demonstrates that "more data" is not always beneficial in multi-task UDA; rather, the quality and alignment of gradients are paramount. By using saliency map skewness as a proxy for gradient conflict, the SM-DSB provides an effective, label-free way to mitigate negative transfer. Future work will explore additional statistical metrics and the application of this technique to 2D image domain adaptation.
Summary updated