Abstract
Noise-Aware Generalization (NAG) addresses the simultaneous presence of in-domain label noise and out-of-domain distribution shifts. Traditional methods for Learning with Noisy Labels (LNL) and Domain Generalization (DG) often fail when these challenges overlap, as distribution shifts can be mistaken for noise and vice versa. We propose Domain Labels for Noise Detection (DL4ND), a framework that identifies noisy samples by comparing them against class proxies in different domains. Our experiments across seven datasets demonstrate that DL4ND significantly outperforms state-of-the-art methods, providing up to a 12.5% accuracy boost.
What Problem Does This Paper Address?
In real-world machine learning applications, data quality is often compromised by two distinct factors: incorrect labels (Learning with Noisy Labels, LNL) and shifts in data distribution (Domain Generalization, DG). While these problems have been studied extensively in isolation, they frequently co-occur in practice.
The authors identify a "catch-22" called the NAG Challenge: distribution shifts caused by a change in domain (which the model should learn) look remarkably similar to distribution shifts caused by label noise (which the model should ignore). Standard DG methods fail because they try to fit the noise, while standard LNL methods fail because they mistake difficult-to-learn domains for noise, leading to poor generalization.
Methods
The proposed solution, Domain Labels for Noise Detection (DL4ND), leverages the intuition that spurious features (e.g., background color) are often domain-specific. A noisy sample might appear "clean" within its own domain due to these features, but it will likely look very different from clean samples in other domains.
The DL4ND framework operates in three main stages:
- Warmup: The model is trained briefly to learn basic features.
- Sample Separation: Using a Gaussian Mixture Model (GMM) on the loss distribution, the model splits data into "low-loss" (likely clean) and "high-loss" (likely noisy) groups.
- Cross-Domain Relabeling: The model creates "proxies" (average feature representations) for every class in every domain using the clean samples. For a high-loss sample in one domain, the model looks for the closest class proxy in a different domain. By forcing a cross-domain comparison, the model ignores domain-specific spurious features and focuses on intrinsic, class-defining features to find the correct label.
Key Results and Insights
The authors evaluated DL4ND across seven diverse datasets, including real-world noise (VLCS, CHAMMI-CP) and synthetic benchmarks (PACS, OfficeHome, DomainNet).
- Superior Performance: DL4ND outperformed 12 state-of-the-art DG and LNL methods. On the OfficeHome dataset with 60% noise, it provided a 12.5% improvement in accuracy over the baseline.
- Biomedical Success: On CHAMMI-CP, a complex biomedical dataset with high natural noise, DL4ND was the only method that provided a consistent average boost across all settings.
- The "Spurious Feature" Trap: Within a single domain (e.g., "Photos"), a model might think "brown background" means "lion." Cross-domain comparison forces the model to look at the actual shape of the animal, as sketches or other domains won't share that specific background noise.
Conclusion
DL4ND effectively untangles label noise from domain shifts by utilizing cross-domain information. While the method currently requires explicit domain labels, future research could explore latent domain discovery to broaden its applicability. The significant performance gains across diverse datasets highlight the importance of noise-aware strategies in building robust machine learning models.
Summary updated