Uncertainty-driven Affordance Discovery for Efficient Robotics Manipulation

Pietro Mazzaglia, Taco Cohen, Daniel Dijkman

TL;DR

Information-driven Affordance discovery (IDA) is a novel framework that enables robots to learn "affordances"—the possible actions an environment supports—through highly efficient, self-directed exploration. By using an information-theoretic measure to target interactions that maximize knowledge gain, IDA allows robots to master complex tasks like grasping and opening drawers with orders of magnitude less data than traditional supervised or reinforcement learning methods.

What Problem Does This Paper Address?

In robotics, an affordance is a map of the environment that tells the agent which actions (like grasping, pulling, or stacking) are likely to succeed at specific locations. Traditionally, learning these maps requires massive, expensive datasets—either millions of simulated trials or thousands of hours of human demonstrations.

Even with these datasets, models often fail to generalize to new objects or environments. The authors address the challenge of data efficiency: How can a robot discover these affordances on its own, in a new environment, without needing a massive pre-existing dataset?

Methods

The authors frame affordance discovery as a contextual bandit problem. Instead of randomly poking around, the robot chooses actions that are both likely to succeed and likely to provide new information.

The IDA Framework

  1. Architecture: The system uses a U-Net-based auto-encoder. It takes a 2D point cloud of the scene as input and outputs an "affordance map"—a pixel-wise probability of success for a specific action primitive (e.g., a grasp at a certain orientation).
  2. Uncertainty via Ensembles: To measure what the robot doesn't know, the model uses an ensemble of multiple decoders sharing a single encoder.
  3. Information Radius (JSD): The core innovation is using the Jensen-Shannon Divergence (JSD), also known as the information radius, to quantify uncertainty. It measures the disagreement between the ensemble members.
  4. Directed Exploration: During training, the robot follows an Upper Confidence Bound (UCB) strategy. It picks actions that maximize a combination of the expected reward (success) and the information gain (uncertainty). This drives the robot to "experiment" in areas where its model is most unsure.
Fig. 1: Information-driven affordance discovery. The model processes inputs from the environment (2D point cloud) using a single encoder, concatenates action parameters and decodes visual affordance maps with multiple decoders (ensemble). Averaging the outputs of these networks, we can extract reliable affordance maps, thanks to the ensemble diversity. Computing the information radius, we can obtain information gain maps about affordances in the scene, to drive considerate explorative interactions. Images represent actual model outputs from IDA in the ManiSkill2 Open Drawer environment.

Datasets

The researchers evaluated IDA in both simulation and the real world:

  • ManiSkill2 Benchmark: A high-fidelity simulator used for tasks including grasping (Cube, YCB, and EGAD object sets), stacking cubes, and opening drawers.
  • Real-World Setup: A UFACTORY xArm 6 robot equipped with a wrist-mounted RGBD camera. The robot was tasked with learning to grasp four distinct toy objects (a duck, a bowling pin, a ball, and a die) from scratch.

Key Insights and Ideas

  • Active Learning vs. Big Data: The paper demonstrates that well-directed, interactive learning can outperform models pre-trained on massive synthetic datasets (like Contact-GraspNet). By interacting with the specific objects at hand, the robot learns more relevant features.
  • The Power of Disagreement: Using the JSD to find where ensemble members disagree provides a precise "map of ignorance." This allows the robot to ignore empty space or already-mastered areas and focus its physical "thinking" on the edges and textures of complex objects.
  • Balancing Curiosity and Success: Pure curiosity (sampling only for information) can be slow. By using UCB to include a reward term, the robot first consolidates "easy" affordances, which provides a stable foundation for exploring more difficult interactions later.

Key Results

IDA significantly outperformed standard baselines (Random, Greedy, and Where2Act) across all tested categories.

  • Simulation: In the ManiSkill2 benchmark, IDA reached the highest mean success rates. Notably, on the difficult "Open Drawer" task, IDA achieved nearly 100% success, while the Greedy baseline struggled at around 73%.
  • Real-World: The robot achieved a 90% grasping success rate after only a few hundred interactions. As shown in the results below, IDA learns significantly faster than random sampling or the Where2Act baseline.
Real-world grasping results show IDA reaching 90% success, far outpacing the baselines.
  • Data Efficiency: The authors found that interactive learning methods eventually outperformed pre-training on 17.7 million simulated grasps, proving that "learning by doing" in the target environment is superior to massive general pre-training.

Limitations and Future Work

  • Reliance on Primitives: The current system relies on pre-defined motion planning for action primitives (e.g., "move to point X and close gripper"). It does not yet learn the low-level motor control from scratch.
  • Early-Stage Noise: In the very first few interactions (under 50), IDA can sometimes perform worse than random sampling because its uncertainty estimates haven't "settled" yet.
  • Long-Horizon Tasks: While IDA is excellent at discovering single-step affordances, the authors aim to extend this to hierarchical controllers or Large Language Models (LLMs) to sequence these discovered actions into complex, multi-stage plans.

Summary updated

Abstract

Robotic affordances, providing information about what actions can be taken in a given situation, can aid robotic manipulation. However, learning about affordances requires expensive large annotated datasets of interactions or demonstrations. In this work, we argue that well-directed interactions with the environment can mitigate this problem and propose an information-based measure to augment the agent's objective and accelerate the affordance discovery process. We provide a theoretical justification of our approach and we empirically validate the approach both in simulation and real-world tasks. Our method, which we dub IDA, enables the efficient discovery of visual affordances for several action primitives, such as grasping, stacking objects, or opening drawers, strongly improving data efficiency in simulation, and it allows us to learn grasping affordances in a small number of interactions, on a real-world setup with a UFACTORY XArm 6 robot arm.
BibTeX
@inproceedings{mazzaglia2024uncertainty,
  author = {Pietro Mazzaglia and Taco Cohen and Daniel Dijkman},
  title = {{Uncertainty-driven Affordance Discovery for Efficient Robotics Manipulation}},
  booktitle = {IEEE International Conference on Robotics and Automation},
  year = {2024},
}
Figure