Abstract
A commercial robot, trained by its manufacturer to recognize a predefined number and type of objects, might be used in many settings, that will in general differ in their illumination conditions, background, type and degree of clutter, and so on. Recent computer vision works tackle this generalization issue through domain adaptation methods, assuming as source the visual domain where the system is trained and as target the domain of deployment. All approaches assume to have access to images from all classes of the target during training, an unrealistic condition in robotics applications. We address this issue proposing an algorithm that takes into account the specific needs of robot vision. Our intuition is that the nature of the domain shift experienced mostly in robotics is local. We exploit this through the learning of maps that spatially ground the domain and quantify the degree of shift, embedded into an end-to-end deep domain adaptation architecture. By explicitly localizing the roots of the domain shift we significantly reduce the number of parameters of the architecture to tune, we gain the flexibility necessary to deal with subset of categories in the target domain at training time, and we provide a clear feedback on the rationale behind any classification decision, which can be exploited in human-robot interactions. Experiments on two different settings of the iCub World database confirm the suitability of our method for robot vision.
Introduction
In robotics, "domain shift" occurs when a model trained in a controlled environment (source domain) is deployed in a real-world setting (target domain) with different lighting, backgrounds, or clutter. Traditional Domain Adaptation (DA) techniques often assume access to unlabeled data from all target classes during training—a "transductive" approach that is often impractical for robots encountering new environments.
The core insight of this work is that domain shift in robotics is often local. While the object of interest remains constant, the background or lighting changes. Standard DA methods attempt to align global distributions, which can fail if the robot only sees a subset of object categories in the new environment (the "sub-target" scenario).
The LoAd Architecture
The proposed LoAd (Local Adaptive) network addresses domain shift by spatially localizing it.
Domain Localization and Maps
The network utilizes a Domain Localization Network to generate "domainness maps." These maps distinguish between domain-specific regions (e.g., a unique lab background) and domain-generic regions (the object itself). This process is inspired by visualization techniques like Grad-CAM but repurposed for identifying the roots of domain variance.
Multiplicative Fusion and Adaptive Pooling
The architecture duplicates feature maps from the final convolutional layer. One set is element-wise multiplied by the domainness maps, effectively "attending" the network's focus toward shared, domain-invariant features. To further increase robustness, the model employs adaptive max pooling, reducing the feature map to a fixed 3x3 size. This reduces parameter count in the fully connected layers and improves invariance to the object's spatial location.
Experimental Setup
The method was evaluated using the iCubWorld Transformations dataset, which features 15 object categories filmed by a robot. The dataset captures two primary types of domain shift:
- Translation: Objects move in a semi-circle, altering background and lighting.
- Scale: Objects move toward or away from the camera, changing their relative size and background visibility.
Results and Discussion
LoAd was compared against state-of-the-art methods like Domain-Adversarial Neural Networks (DANN) and Auto-DIAL.
- Translation Performance: LoAd achieved 64.55% accuracy, significantly higher than DANN (51.08%) and Auto-DIAL (59.59%).
- Robustness to Missing Classes: In the "sub-target" test (where only 8 of 15 classes were available in the target domain during training), LoAd's performance remained stable (63.82%), whereas global alignment methods typically collapse.
- Scale Challenges: While LoAd excelled in translation, scale changes proved more difficult. When an object is very close, distinguishing between the object and the shared background becomes harder as the object dominates the frame.
Conclusion
By explicitly localizing domain shift, the LoAd network provides a more robust and explainable solution for robot vision. It successfully handles scenarios where only a subset of categories is available for adaptation, a common hurdle in real-world deployment. Future work will explore "open set" adaptation and the integration of multi-modal data like RGB-D to further improve localization in complex environments.
Summary updated