Abstract
Standard deep learning classifiers often struggle with "domain shift," where test environments differ significantly from training data. This paper proposes a shift from direct classification to similarity learning using Siamese networks. By training a model to learn similarity functions rather than class labels, robots can achieve better generalization. Our results show that comparing new images to a few random training examples reduces classification errors by up to 64% across diverse datasets, including a novel robot-collected door dataset.
Introduction
Deep learning typically assumes a "stable domain," where training data encompasses all possible variations encountered at test time. However, for robots operating "in the wild," this assumption is frequently violated. A robot trained to recognize open doors in one facility may fail in another due to changes in lighting, wall color, or door materials.
The core issue is that direct CNN classifiers are prone to overfitting to spurious correlations. If a training set for "open doors" consistently features a specific background color, the network may incorrectly associate that color with the "open" state. This phenomenon, known as domain shift, causes the network to ignore robust discriminative features in favor of environment-specific details.
Methods
To address these challenges, we propose a methodology centered on similarity learning rather than direct label prediction.
Siamese Architecture
We utilize a Siamese network architecture, typically based on a VGG16 backbone, featuring two identical branches with shared weights. Unlike standard classifiers that output class probabilities for a single image, this network processes a pair of images and predicts a similarity score $d_c$ ranging from 0 to 1.
Learning the Embedding
By training on image pairs, the network learns a high-dimensional embedding space where images of the same class are clustered together and different classes are separated. This approach forces the model to identify what makes two objects "the same," focusing on intrinsic features rather than memorizing the appearance of specific classes.
Classification by Sampling
At test time, a query image $Q$ is classified by comparing it to a small set of $k$ random exemplars from the training set for each class. The similarity scores are averaged, and the class with the highest average similarity is selected. This "sampling" approach is computationally efficient and avoids the need for exhaustive comparisons.
Datasets and Evaluation
The proposed method was evaluated across four distinct datasets to demonstrate its robustness:
- Open and Closed Doors: A new RGB-D dataset collected by a mobile robot (shown below) across two different warehouses. This dataset specifically tests domain shift by training in one location and testing in another.
- NYU2: RGB-D indoor scenes used for distinguishing between "office" and "home office" environments.
- Pasadena Street Trees: RGB images of tree species from Google Streetview, characterized by high background variability.
- Omniglot: Handwritten characters used for testing generalization across writers (INST) and to entirely new symbols (SYMB).
Key Insights
- Global Embedding Properties: Siamese networks learn an approximately global embedding, preserving distance relationships even across arbitrary data subsets.
- Robustness of Random Sampling: High performance is achieved even with a small number of random exemplars ($k=10$), suggesting that "perfect" representative images are not required.
- Mitigating Spurious Features: Pair-based learning naturally filters out accidental correlations (like background color) because such features are unlikely to be consistent across randomly sampled pairs.
Results
The similarity-based approach consistently outperformed standard classification baselines:
- Error Reduction: Classification errors were reduced by 28% to 64% across all tested datasets.
- Robot Door Task: Accuracy on the challenging door dataset improved from a baseline of 86.1% to over 91%.
- Omniglot SYMB: In the most difficult scenario—recognizing symbols never seen during training—the Siamese approach achieved ~81.5% accuracy, significantly higher than the ~72% achieved by the baseline.
- Comparative Performance: Our method outperformed specialized few-shot learning algorithms like "Learning to Compare" (LtC), even when those algorithms were provided with more exemplars.
Conclusion
Moving from direct classification to similarity learning offers a powerful way to handle domain shift in robotic vision. By focusing on relative similarity rather than absolute labels, models become more robust to the "intra-class variation" found in real-world environments. Future work will explore "learning the consensus"—using secondary networks to optimize how similarity scores from multiple exemplars are combined.
Summary updated