This paper, originally presented at ICRA 2021, addresses a critical reality check for robotic vision: the intersection of Open World Recognition (OWR) and Domain Shift. While OWR allows robots to identify unknown objects and learn them incrementally, most research assumes a static environment. This study breaks that assumption, demonstrating that current OWR algorithms struggle significantly when transitioning from controlled simulations to unconstrained real-world environments.
The Challenge: Breaking Assumptions
In robotic vision, researchers often operate under two restrictive assumptions:
- The Closed World Assumption (CWA): The robot will only encounter object categories it was trained on.
- The Closed Domain Assumption (CDA): The training data and deployment environment share identical visual characteristics (lighting, camera angles, etc.).
While OWR research has made strides in breaking the CWA—enabling robots to reject unknown objects—it frequently ignores the CDA. In practice, a robot trained on synthetic images (Source Domain) will eventually be deployed in a cluttered, poorly lit room (Target Domain). The authors argue that OWR and Domain Shift must be studied together because a robot can easily mistake a "known object in a new environment" for a "new, unknown object."
Methodology and Benchmarking
The authors evaluated three state-of-the-art OWR algorithms:
- NNO (Nearest Non-Outlier): A non-parametric approach using class centroids.
- DeepNNO: An end-to-end deep learning extension of NNO.
- B-DOC (Boosting Deep OWR by Clustering): A method using global and local clustering to define class-specific rejection thresholds.
To combat domain shift, they integrated three Domain Generalization (DG) strategies: RSDA (advanced data augmentation), Relative Rotations (self-supervised structural learning), and Self-Challenging (SC) (a regularization technique that masks informative features to force robustness).
Testing was conducted across three datasets: synROD (synthetic 3D renders), ROD (clean turntable images), and ARID (real-world autonomous robot data with occlusions and challenging lighting). Scenarios included Synthetic-to-Real (synROD → ROD/ARID) and Constrained-to-Unconstrained (ROD → ARID) transitions.
Key Results and Performance Collapse
The study revealed a dramatic performance collapse when moving from synthetic to real-world domains. For instance, moving from synROD to the unconstrained ARID caused OWR performance (measured by Harmonic Mean) to drop by nearly 45% for some models.
Key insights include:
- Brittleness of Deep OWR: Surprisingly, the non-end-to-end NNO often outperformed deep models under heavy domain shift. Deep models tend to overfit to the source domain's style, making their "unknown" thresholds unreliable in new environments.
- Domain Shift Mimics Novelty: Visual discrepancies in new environments are often so high that models reject known categories (like a mug) as "unknown."
- The Gap Remains: While the Self-Challenging (SC) strategy provided the most significant boost, a 30-40% performance gap remains compared to in-domain testing. Simply "plugging in" existing DG methods is insufficient for robust OWR.
Conclusion
The authors conclude that disentangling domain-specific features from semantic features remains an open challenge. Future work should explore Online Domain Adaptation, where a robot updates its understanding of the environment (domain) simultaneously while learning new object categories.
Summary updated