Introduction
In machine learning, the assumption that training data perfectly represents the real-world distribution is rarely met. Instead, models frequently encounter "sampling errors" that lead to significant performance degradation during deployment. Researchers from Seoul National University have identified that these errors typically manifest in two distinct ways: Domain Generalization (DG) and Unbiased Learning (UBL). While existing research often treats these as separate problems, this paper introduces a unified framework called Instance-wise Adaptive Inference (IAI) to address both simultaneously by dynamically adapting to distribution shifts at test time.
The Challenge of Sampling Errors
Sampling errors generally fall into two categories:
- Single Domain Generalization (SDG): This occurs when a model is trained on one domain (e.g., photos) but must generalize to unseen domains (e.g., sketches or cartoons) where the visual style differs significantly.
- Unbiased Learning (UBL): This occurs when a model learns spurious correlations present in the training data. For instance, if a dataset primarily features young women and old men, a gender classifier might incorrectly learn to associate age-related features (like wrinkles) with the male class.
Most current solutions are over-optimized for one specific type of error. A model designed to be invariant to image style (SDG) often fails to ignore biased attributes (UBL), and vice versa. The authors argue that because the specific nature of the distribution shift is often unknown at deployment, a more flexible, conservative approach is required.
Proposed Methodology: Instance-wise Adaptive Inference (IAI)
The IAI framework employs a three-step strategy to achieve unified robustness:
1. Widening the Feature Space
To prevent the model from over-relying on the narrow set of features present in the source data, the framework uses a style generation module ($G$). This module creates diversely stylized versions of the training images, forcing the model to consider a broader range of potential semantic features.
2. Feature Disentanglement
The model utilizes a Disentangling Module ($F_b$) and a cosine similarity loss to ensure that different sub-networks learn independent semantic representations. By separating features like texture, shape, and color into distinct modules, the system can later adjust their relative importance.
3. Instance-wise Adaptive Inference
This is the core innovation. At test time, the model monitors the activation levels of each disentangled feature module for a specific input. If a feature that was less prominent during training (e.g., "shape") becomes highly activated for a target image, the model infers that this feature is likely more reliable for that specific instance and increases its weight in the final prediction.
Experimental Evaluation
The framework was evaluated on several benchmarks:
- PACS & DG-CIFAR: Benchmarks for Single Domain Generalization involving Photo, Art, Cartoon, and Sketch domains.
- B-CIFAR & IMDB: Datasets designed to test Unbiased Learning, where B-CIFAR contains synthetic biases and IMDB features real-world gender classification biased by age.
Key Results
IAI demonstrated superior consistency and generalization across all tasks:
- PACS (SDG): IAI achieved 57.40% average accuracy, significantly outperforming the baseline (33.38%) and specialized SDG models like ADA (50.85%).
- IMDB (UBL): IAI reached 81.03% accuracy, surpassing specialized unbiased learning models such as LfF (77.09%) and UBNet (79.66%).
Conclusion
The primary insight of this work is that specialization breeds fragility. By focusing on feature disentanglement and test-time activation as a proxy for feature importance, the IAI framework provides a robust defense against diverse sampling errors. While the authors note that adaptive re-weighting is less effective when distribution shifts are minimal, the framework represents a significant step toward models that can safely navigate the unpredictable shifts of real-world data deployment.
Summary updated