Introduction: The Appearance Trap
Most modern object detectors are designed for a "closed-world" setting, where they are trained to recognize a specific list of categories. In real-world applications, such as robotics or autonomous driving, systems encounter thousands of objects they were never explicitly trained to see. Standard RGB-based models often fail in these scenarios because they overfit to the textures and colors of their training data—a phenomenon known as the "appearance trap." If a model is trained only on vehicles, it may fail to recognize a novel object simply because it does not share the visual features of a car.
The GOOD Framework
The Geometry-guided Open-world Object Detector (GOOD) addresses this by leveraging the fact that while objects vary wildly in color, they share fundamental geometric properties like shape, volume, and boundaries. The authors propose a two-phase framework:
- Geometric Proposal Generation: Instead of raw pixels, the system uses monocular estimators to generate depth and surface normal maps. An Object Proposal Network (OPN) is trained on these geometric maps. Because geometry abstracts away distracting textures, the network learns a more "generic" sense of what constitutes an object.
- Pseudo-Labeling and Training: This geometry-trained network "pseudo-labels" unannotated objects in the training set. A final RGB-based detector is then trained using a combination of original manual labels and these new geometric pseudo-labels, forcing the model to learn that "novel-looking" things are still objects.
Key Insights and Results
The researchers found that geometric cues (depth and surface normals) are significantly more class-agnostic than RGB signals. By focusing on "where" and "how big" rather than "what color," the model avoids relying on specific textures.
Performance Gains
- Recall Improvement: When trained on limited classes, GOOD achieved a 24% relative improvement in recall (AR@100) over previous state-of-the-art methods.
- Efficiency: GOOD generalizes better with fewer training classes; a model trained on 39 classes performs as well as a standard RGB model trained on 80 classes.
- Cross-Dataset Robustness: In tests on the ADE20K dataset, GOOD consistently outperformed baselines, proving that geometric cues remain stable even when moving between entirely different image distributions.
Conclusion
While RGB is effective for detecting small objects with fine details, geometry is superior for identifying large, novel shapes. By combining these modalities through pseudo-labeling, GOOD provides a robust solution for open-world object detection, moving computer vision closer to the generalized perception required for real-world autonomy.
Summary updated