Abstract
Standard Convolutional Neural Networks (CNNs) often struggle with Out-of-Distribution (OOD) generalization, frequently relying on spurious correlations between objects and their backgrounds. This paper introduces a robust classification framework that adapts the Hypercolumns technique to a ResNet-18 architecture. By integrating low-level structural features from early layers with high-level semantic features from deeper layers, the model learns invariant representations that focus on the object of interest rather than its context. Evaluated on the NICO dataset using a leave-one-domain-out protocol, the proposed method demonstrates significant accuracy improvements and superior object localization in saliency maps compared to baseline models.
Introduction
Deep learning models typically achieve high accuracy when test data is Independent and Identically Distributed (IID) relative to training data. However, they often fail in Domain Generalization (DG) scenarios. A primary cause is the learning of "spurious correlations"—for instance, a model might associate a blue background with an airplane rather than learning the airplane's physical structure. This research aims to force models to learn invariant features—physical characteristics that remain constant across different environments, such as the four legs of a horse regardless of whether it is in snow or on a farm.
Methodology: Hypercolumn Adaptation
The authors leverage the concept of Hypercolumns, originally used for image segmentation, to enhance classification robustness. Unlike standard CNNs where classification relies solely on the final layer, this architecture preserves "local" information from earlier layers to prevent the loss of structural detail.
The proposed pipeline involves several key stages:
- Feature Extraction: Extracting feature maps from multiple intermediate convolutional layers and residual blocks of a ResNet-18 backbone.
- Processing: Each map undergoes a $1 \times 1$ convolution followed by bilinear interpolation for upsampling.
- Fusion: To capture class-level essence without pixel-level noise, maps are passed through a $56 \times 56$ MaxPool2D layer.
- Classification: These features are concatenated into a single "hyper vector" and processed by a final fully connected layer with a softmax activation.
By simultaneously processing low-level geometric features (edges, shapes) and high-level semantic concepts, the model effectively disentangles objects from their backgrounds.
Experimental Setup
The method was evaluated using the NICO (Non-I.I.D. Image Classification with Objects) dataset, which is specifically designed for OOD tasks. NICO contains 19 classes across various contexts (e.g., "dog on grass" vs. "dog in water"). The authors employed a "leave-one-domain-out" protocol, training on a subset of contexts and testing on entirely unseen ones to measure true generalization performance.
Results and Discussion
The Hypercolumn-adapted model consistently outperformed the baseline ResNet-18 across varying levels of difficulty. In tests where 7 contexts were left out, the proposed model achieved 83.0% accuracy, a 4.1% improvement over the baseline.
Qualitative analysis using saliency maps confirms that the model focuses more tightly on the actual object (e.g., the cat or car) while the baseline is often distracted by background elements like snow, water ripples, or grass. This suggests that the inclusion of early-layer features provides a "geometric anchor" that prevents the model from over-relying on contextual cues.
Conclusion
This work demonstrates that multi-layer representation learning via Hypercolumns significantly improves OOD robustness without requiring complex data augmentation or adversarial training. Future research may explore scaling this approach to larger architectures like Vision Transformers or incorporating attention mechanisms to dynamically weight the importance of different intermediate layers.
Summary updated