Abstract
Recent visual pose estimation and tracking solutions provide notable results on popular datasets such as T-LESS and YCB. However, in the real world, we can find ambiguous objects that do not allow exact classification and detection from a single view. In this work, we propose a framework that, given a single view of an object, provides the coordinates of a next viewpoint to discriminate the object against similar ones, if any, and eliminates ambiguities. We also describe a complete pipeline from a real object's scans to the viewpoint selection and classification. We validate our approach with a Franka Emika Panda robot and common household objects featured with ambiguities. We released the source code to reproduce our experiments.
Introduction
In controlled laboratory environments, objects are typically distinct and easily separable. However, real-world scenarios, such as grocery store shelves, present significant visual ambiguity. For instance, two different cereal boxes may appear identical from the side or back, with distinguishing features only visible on the front panel. Standard static computer vision classifiers struggle in these cases because training on identical-looking images with different labels leads to poor convergence or unreliable "guessing."
To address this, we propose an active perception framework. Instead of relying on a single static viewpoint, the system identifies when a view is ambiguous and intelligently moves the camera to a "next best view" (NBV) that contains discriminative features.
Methodology
Our pipeline integrates Augmented Autoencoders (AAE) with an active planning strategy to handle both pose estimation and classification under ambiguity.
Learning and Ambiguity Ranking
We utilize an AAE to map object images into a low-dimensional latent space (embeddings). These embeddings capture essential geometric and visual features. By comparing embeddings of different objects across all possible rotations, we calculate an "ambiguity rank." If a viewpoint of Object A is highly similar to a viewpoint of Object B in the latent space, that orientation is marked as ambiguous and unreliable for classification.
System Pipeline
The framework consists of two main phases: training and inference. During training, we use the ambiguity rank to prune the dataset, ensuring the classifier is only trained on discriminative views. This prevents the model from being confused by identical-looking surfaces.
During inference, the robot first estimates the object's pose using the AAE. If the current pose's ambiguity rank exceeds a threshold, the system calculates an NBV. This calculation accounts for the robot's physical reachability and potential collisions, selecting the least ambiguous reachable position to perform the final classification.
Experimental Results
We validated the system using a Franka Emika Panda robot and a custom dataset of household objects, including baby food boxes and milk bottles with one-sided ambiguities.
- Classification Accuracy: The system achieved a 91% accuracy rate across 230 real-world experiments. Out of these, only 20 resulted in incorrect classifications.
- Active Perception Efficiency: The active strategy significantly outperformed random movement baselines, reaching correct classifications faster by intentionally seeking out discriminative features.
- Robustness: The AAE-based similarity metric proved stable under varying lighting conditions, outperforming traditional pixel-wise or SIFT-based comparisons.
Conclusion
This work demonstrates that active perception is crucial for handling real-world visual ambiguity. By explicitly modeling ambiguity and planning camera movements, robots can reliably classify objects that are indistinguishable from certain angles. Future work will focus on reducing the dependency on high-quality 3D scans and extending the framework to handle complex scenes with significant occlusions.
Summary updated