Active Perception for Ambiguous Objects Classification

Evgenii Safronov, Nicola A. Piga, Michele Colledanchise, Lorenzo Natale

TL;DR

This paper introduces an active perception framework that allows robots to classify "ambiguous" objects—items that look identical from certain angles but different from others (like two flavors of the same brand of cereal). By calculating an "ambiguity rank" using latent embeddings, the system determines if a current view is unreliable and autonomously moves the camera to the "next best view" to ensure accurate classification.

What Problem Does This Paper Address?

In a perfect lab setting, objects are usually distinct. In the real world, however, we encounter visual ambiguity. Think of a grocery store shelf: two boxes of pasta might have identical sides and backs, with the only difference being a small "Spaghetti" vs. "Linguine" label on the front.

Standard computer vision classifiers struggle here for two reasons:

  1. Training Conflict: If you train a model on images that look identical but have different labels (e.g., the backs of the boxes), the model may fail to converge or simply "guess," leading to poor reliability.
  2. Static Limitation: A robot looking at the back of the box from a single static viewpoint literally cannot know which flavor it is.

The authors tackle this by asking: How can a robot automatically identify which views are useless for training, and how can it intelligently move its camera to find a view that actually matters?

Methods

The researchers proposed a pipeline that combines Augmented Autoencoders (AAE) with an active planning strategy.

  • Learning Embeddings: They use an AAE to map object images into a low-dimensional "latent space" (embeddings). Because the AAE is trained to reconstruct clean views from noisy ones, these embeddings capture the essential geometric and visual features of the object.
  • The Ambiguity Rank: This is the core innovation. By comparing the embeddings of Object A and Object B across all possible rotations, they calculate a similarity score. If the best match for a view of Object A is a nearly identical view of Object B, that orientation is assigned a high "ambiguity rank."
  • Smart Training: Instead of feeding every image into a classifier (like ResNet-18), they use the ambiguity rank to filter the dataset. They only train the classifier on "discriminative" views—those where the objects actually look different.
  • Active Inference: During operation, the robot first estimates the object's pose. It then checks the ambiguity rank for that pose. If the view is too ambiguous (e.g., looking at the back of the box), the robot calculates a "Next Best View" (NBV). It moves the camera to a reachable position that is predicted to have the lowest possible ambiguity, then tries the classification again.

Datasets

Since standard datasets like YCB or T-LESS don't focus on this specific type of "one-sided" ambiguity, the authors created their own:

  • Modified YCB Mustard Bottle: A digital twin of the standard YCB mustard bottle, but with a modified texture on one side to create a controlled ambiguous pair.
  • Real-World Household Objects: They used a 3D scanner to create high-fidelity models of two groups of objects:
  • Food Boxes: Different flavors of "Mellin" baby food that look identical except for the front face.
  • Bottles: "Humana" milk bottles with similar shapes but different labels.

Key Insights and Ideas

  • Data Pruning is Better than Brute Force: The authors found that training on the entire dataset actually hurt performance. By explicitly excluding ambiguous views from the training set, the classifier became much more "confident" and accurate on the views that actually contained distinguishing features.
  • Pose Estimation as a Prerequisite: You can't know if a view is ambiguous unless you have a rough idea of the object's orientation. The system uses the AAE to estimate pose first, then uses that pose to look up the pre-calculated ambiguity of that specific angle.
  • The "Next Best View" isn't just the "Best" View: The robot has to account for its own physical constraints (reachability) and potential collisions. The algorithm selects the least ambiguous view that the robot can actually reach from its current position.

Key Results

  • Classification Accuracy: In real-robot experiments using a Franka Emika Panda arm, the system achieved a 91% classification accuracy. Out of 230 experiments, only 20 resulted in incorrect classifications.
  • Active vs. Random: The active perception strategy significantly outperformed a "random move" baseline. By intentionally seeking out discriminative features, the robot reached a correct conclusion much faster and more reliably.
  • Robustness to Lighting: The authors demonstrated that their similarity metric (based on AAE embeddings) remained stable even when light intensity changed, making it more robust than simple pixel-wise comparisons or SIFT descriptors.

Limitations and Future Work

While successful, the paper notes a few hurdles:

  • Dependence on 3D Models: The current pipeline requires high-quality 3D scans of the objects to pre-calculate the ambiguity ranks and train the AAE.
  • Symmetry Issues: Objects with high geometric symmetry (like the bottles) are harder for the AAE to orient correctly, which can lead to errors in the ambiguity estimation.
  • Occlusions: The current work assumes a relatively clear view of the object. Future research will need to address how the robot should move when the "best" view is blocked by another object.

The authors plan to extend this work to handle more complex scenes and investigate how this similarity metric can be used to automatically cluster groups of ambiguous objects within larger datasets.

Summary updated

Abstract

Recent visual pose estimation and tracking solutions provide notable results on popular datasets such as T-LESS and YCB. However, in the real world, we can find ambiguous objects that do not allow exact classification and detection from a single view. In this work, we propose a framework that, given a single view of an object, provides the coordinates of a next viewpoint to discriminate the object against similar ones, if any, and eliminates ambiguities. We also describe a complete pipeline from a real object’s scans to the viewpoint selection and classification. We validate our approach with a Franka Emika Panda robot and common household objects featured with ambiguities. We released the source code to reproduce our experiments.
BibTeX
@inproceedings{safronov2021active,
  author = {Evgenii Safronov and Nicola A. Piga and Michele Colledanchise and Lorenzo Natale},
  title = {{Active Perception for Ambiguous Objects Classification}},
  booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems},
  year = {2021},
  url = {https://openreview.net/forum?id=m0hK5zk9Eh},
}

Loading related content...

Figure