Introduction
Deep learning models often struggle with domain shift, where the data encountered during testing differs from the training distribution. While Domain Generalization (DG) techniques aim to mitigate this, most traditional methods rely on explicit "domain labels" (e.g., knowing which specific camera or sensor produced a sample). In real-world scenarios, these labels are frequently unavailable or the data contains hidden mixtures of domains. This paper introduces Domain-Free Domain Generalization (DFDG), a model-agnostic approach designed to learn robust, domain-invariant features without requiring domain-specific metadata.
The DFDG Framework
DFDG employs two primary strategies to focus the model on core class features while ignoring domain-specific noise:
- Class-Aware Alignment (Regularization): Rather than aligning high-dimensional intermediate features, DFDG operates in the label space. It calculates class centroids based on the model's soft-label probability outputs across a training batch. By penalizing samples that deviate significantly from their class's average distribution, the model is encouraged to learn consistent class relationships that persist across different domains.
- Saliency-Guided Masking (Augmentation): To prevent overfitting to superficial features (like image backgrounds or transient sensor noise), the framework utilizes SmoothGrad saliency maps. These maps identify input regions least critical for classification. DFDG then shuffles these "unimportant" segments during training, forcing the model to rely on robust, discriminative features.
Key Insights
- Label Space vs. Feature Space: Aligning in the final output probability space is more efficient and less noisy than aligning hidden layers, as it directly captures the essential relationships between classes.
- Proactive Saliency: While typically used for explainability (XAI), saliency maps here serve as a training tool to actively "mute" potential sources of domain-specific overfitting.
- Cross-Modal Versatility: The techniques developed for computer vision translate effectively to time-series sensor data, helping models ignore transient noise in vibration or motion signals.
Experimental Evaluation
The authors evaluated DFDG across three diverse benchmarks: mechanical fault diagnosis (Bearings), human activity recognition (HHAR), and image classification (PACS).
On the HHAR dataset, which involves motion sensor data from various devices, DFDG achieved a significant performance boost. It reached an average accuracy of 58.18%, substantially outperforming the "TrainAll" baseline (45.15%) and other state-of-the-art methods like MMLD.
In the PACS image benchmark, DFDG achieved 82.10% average accuracy, proving competitive with specialized methods like RSC despite not using domain labels. Furthermore, on the Bearings dataset, DFDG demonstrated synergy with existing methods; a hybrid DFDG+RSC model achieved the highest overall accuracy of 84.65%.
Conclusion and Future Work
DFDG provides a robust solution for domain generalization when domain boundaries are unknown. While highly effective, the method introduces additional hyperparameters, such as the masking ratio and alignment weight, which require tuning for specific datasets. Additionally, generating saliency maps adds computational overhead during training. Future research may focus on optimizing these processes and improving saliency map quality for even more precise feature masking.
Summary updated