Learning Relevant Features for Manipulation Skills using Meta-Level Priors

Oliver Kroemer, Gaurav S. SukhatmearXiv 2016 · arXiv

Abstract

This paper presents a novel framework for robotic manipulation that enables autonomous identification of task-relevant object features. By leveraging meta-level priors—learned relationships between geometric features and task relevance from previous skills—the system significantly improves feature selection accuracy. Experimental results on a PR2 robot across six diverse manipulation tasks demonstrate that this approach allows for robust generalization with minimal demonstrations, achieving a 77.8% success rate in novel scenarios.

Introduction

When a robot learns a skill from a human demonstration, it is often overwhelmed by a high-dimensional feature space. For a task like pouring, the robot must distinguish between critical features (e.g., the cup's rim) and irrelevant ones (e.g., the table's color). Traditionally, this "needle in a haystack" problem is solved by manual feature engineering or large-scale statistical learning. This paper addresses the challenge of learning feature relevance autonomously and efficiently by transferring knowledge from previously learned skills.

Methodology

The proposed framework treats feature selection as a transfer learning problem, utilizing "meta-level priors" to guide the robot's attention. The system architecture is divided into several key stages:

  1. Part-Based Feature Generation: Objects are segmented into functional parts using the GrabCut algorithm, focusing on areas of interaction during demonstrations.
  2. Meta-Features and Priors: The core innovation involves calculating "meta-features" that describe the geometric relationship between a feature and the task (e.g., proximity to the hand, alignment with motion). The robot learns a mapping from these meta-features to the probability of relevance, creating a meta-level prior.
  3. Feature Selection and Execution: The system uses Stochastic Search Variable Selection (SSVS), a Bayesian approach, to identify a sparse set of relevant features. This process is initialized with the meta-level prior rather than a uniform distribution. Finally, the motion is executed using Dynamic Movement Primitives (DMPs) adapted to the selected features.
Figure 1: System overview showing the flow from skill demonstrations and meta-level transfer to feature selection and skill execution.

Experimental Setup

The approach was evaluated using a PR2 robot performing six manipulation tasks: placing, pushing, tilting, cutting, pouring, and wiping. The researchers collected 90 demonstrations across three different object sets, primarily utilizing the YCB object set— a standard benchmark for household manipulation. The goal was to test if a robot could generalize these skills to novel objects and configurations with only 15 demonstrations per task.

Results and Discussion

The inclusion of meta-level priors led to significant performance gains:

  • Accuracy: The use of meta-priors more than doubled the average precision and recall in feature selection compared to standard uniform priors.
  • Generalization: The robot showed a superior ability to predict correct goal states in new environments. On the physical PR2 platform, the system achieved a 77.8% success rate across various tasks in novel situations.
  • Transfer Logic: A key insight is that feature relevance is grounded in physical geometry. By encoding these relationships, a robot that has learned to "cut" can apply the logic of what makes a part relevant to a new task like "pouring."
Figure 6: The PR2 robot performing various manipulation tasks, including interacting with bowls, cups, and blocks, to validate the feature selection framework.

Conclusion

By transferring the logic of how to learn relevance rather than just transferring trajectories, this research enables robots to become more autonomous and data-efficient. While the system faced challenges with specific tasks like "pushing" where priors were occasionally misleading, the overall framework demonstrates a powerful path toward robots that can quickly adapt to complex, unstructured human environments. Future work will focus on handling variable object counts and integrating real-time sensory feedback.

Summary updated

Abstract

Robots can generalize manipulation skills between different scenarios by adapting to the features of the objects being manipulated. Selecting the set of relevant features for generalizing skills has usually been performed manually by a human. Alternatively, a robot can learn to select relevant features autonomously. However, feature selection usually requires a large amount of training data, which would require many demonstrations. In order to learn the relevant features more efficiently, we propose using a meta-level prior to transfer the relevance of features from previously learned skills. The experiments show that the meta-level prior more than doubles the average precision and recall of the feature selection when compared to a standard uniform prior. The proposed approach was used to learn a variety of manipulation skills, including pushing, cutting, and pouring.
BibTeX
@misc{kroemer2016learning,
  author = {Oliver Kroemer and Gaurav S. Sukhatme},
  title = {{Learning Relevant Features for Manipulation Skills using Meta-Level Priors}},
  year = {2016},
  eprint = {1605.04439},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1605.04439},
}
Figure