MetaView: Few-shot Active Object Recognition

Wei Wei, Haonan Yu, Haichao Zhang, Wei Xu, Ying Wu

Abstract

In robot sensing scenarios, instead of passively utilizing human captured views, an agent should be able to actively choose informative viewpoints of a 3D object as discriminative evidence to boost the recognition accuracy. This task is referred to as active object recognition. Recent works on this task rely on a massive amount of training examples to train an optimal view selection policy. But in realistic robot sensing scenarios, the large-scale training data may not exist and whether the intelligent view selection policy can be still learned from few object samples remains unclear. In this paper, we study this new problem which is extremely challenging but very meaningful in robot sensing -- Few-shot Active Object Recognition, i.e., to learn view selection policies from few object samples, which has not been considered and addressed before. We solve the proposed problem by adopting the framework of meta learning and name our method "MetaView". Extensive experiments on both category-level and instance-level classification tasks demonstrate that the proposed method can efficiently resolve issues that are hard for state-of-the-art active object recognition methods to handle, and outperform several baselines by large margins.

Introduction

Active Object Recognition (AOR) is a critical capability for autonomous agents, allowing them to interactively explore their environment to resolve visual ambiguity. By moving a camera to capture the most informative viewpoints, a robot can significantly improve its classification accuracy compared to a single, passive observation. However, traditional AOR models are notoriously data-hungry, often requiring thousands of labeled examples per category to learn an effective exploration policy.

In practical robotics, agents frequently encounter novel objects or categories with limited available data. This necessitates a "few-shot" approach to active recognition. The authors introduce the Few-shot Active Object Recognition task, where an agent must learn a generalized view-selection strategy that can be quickly adapted to new object classes using only a handful of "support" examples.

Overview of the Few-shot Active Object Recognition task, showing support sets and query tasks for both category-level and instance-level recognition.

The MetaView Framework

To address this challenge, the researchers proposed MetaView, a framework that combines reinforcement learning with meta-learning. The system is designed to learn an inductive bias—a universal "way of looking"—that transfers across different object types.

Architecture

The MetaView architecture consists of three primary modules:

  1. Perception and Memory: A Convolutional Neural Network (CNN) processes individual 2D views into feature vectors. These features are then fed into a Recurrent Neural Network (RNN), which maintains a temporal state, effectively building a 3D "mental model" of the object as more views are captured.
  2. Active Policy: The agent's next move is determined by a policy network trained via the REINFORCE algorithm. Instead of using raw coordinates, the model utilizes action embeddings to represent camera movements, which enhances the decision-making process.
  3. Meta-Learning Optimization: Using a MAML-style approach, the model is trained on a distribution of tasks. During training, it adapts its parameters based on a small support set and is then optimized to perform well on a query set. This ensures the learned policy is not overfit to specific shapes but is instead a flexible strategy for exploration.
The MetaView architecture: features from views and actions are fused and processed by an RNN to inform a policy network for view selection and a final classifier.

Experimental Results

The framework was evaluated on the ModelNet-40 dataset (rendered into view grids) and the SUN360 panoramic scene dataset. The results demonstrate a significant leap over existing methods:

  • Superior Accuracy: In a 5-way, 1-shot category-level task on ModelNet-40, MetaView achieved 59.77% accuracy, vastly outperforming the previous state-of-the-art "LookAhead" method (38.67%).
  • Fine-Grained Recognition: MetaView proved exceptionally capable at instance-level recognition (distinguishing between two similar airplanes), reaching 94.65% accuracy in 1-shot scenarios.
  • Active Advantage: A key finding was that active exploration is even more vital when data is scarce. MetaView's ability to "learn to look" allowed it to outperform passive, single-view baselines by wide margins.

Conclusion and Future Directions

MetaView represents a paradigm shift in active vision, moving from static policy training to meta-learned exploration strategies. While the current model operates on discretized view grids and occasionally struggles with "long-tail" query instances, it provides a robust foundation for future work. Potential extensions include adapting the framework for continuous 3D movement and implementing lifelong learning capabilities to prevent the forgetting of previously mastered categories.

Summary updated

Abstract

In robot sensing scenarios, instead of passively utilizing human captured views, an agent should be able to actively choose informative viewpoints of a 3D object as discriminative evidence to boost the recognition accuracy. This task is referred to as active object recognition. Recent works on this task rely on a massive amount of training examples to train an optimal view selection policy. But in realistic robot sensing scenarios, the large-scale training data may not exist and whether the intelligent view selection policy can be still learned from few object samples remains unclear. In this paper, we study this new problem which is extremely challenging but very meaningful in robot sensing -- Few-shot Active Object Recognition, i.e., to learn view selection policies from few object samples, which has not been considered and addressed before. We solve the proposed problem by adopting the framework of meta learning and name our method "MetaView". Extensive experiments on both category-level and instance-level classification tasks demonstrate that the proposed method can efficiently resolve issues that are hard for state-of-the-art active object recognition methods to handle, and outperform several baselines by large margins.
BibTeX
@misc{wei2021metaview,
  author = {Wei Wei and Haonan Yu and Haichao Zhang and Wei Xu and Ying Wu},
  title = {{MetaView: Few-shot Active Object Recognition}},
  year = {2021},
  eprint = {2103.04242},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2103.04242},
}
Figure