TL;DR
Information-driven Affordance discovery (IDA) is a novel framework that enables robots to learn "affordances"—the possible actions an environment supports—through highly efficient, self-directed exploration. By using an information-theoretic measure to target interactions that maximize knowledge gain, IDA allows robots to master complex tasks like grasping and opening drawers with orders of magnitude less data than traditional supervised or reinforcement learning methods.
What Problem Does This Paper Address?
In robotics, an affordance is a map of the environment that tells the agent which actions (like grasping, pulling, or stacking) are likely to succeed at specific locations. Traditionally, learning these maps requires massive, expensive datasets—either millions of simulated trials or thousands of hours of human demonstrations.
Even with these datasets, models often fail to generalize to new objects or environments. The authors address the challenge of data efficiency: How can a robot discover these affordances on its own, in a new environment, without needing a massive pre-existing dataset?
Methods
The authors frame affordance discovery as a contextual bandit problem. Instead of randomly poking around, the robot chooses actions that are both likely to succeed and likely to provide new information.
The IDA Framework
- Architecture: The system uses a U-Net-based auto-encoder. It takes a 2D point cloud of the scene as input and outputs an "affordance map"—a pixel-wise probability of success for a specific action primitive (e.g., a grasp at a certain orientation).
- Uncertainty via Ensembles: To measure what the robot doesn't know, the model uses an ensemble of multiple decoders sharing a single encoder.
- Information Radius (JSD): The core innovation is using the Jensen-Shannon Divergence (JSD), also known as the information radius, to quantify uncertainty. It measures the disagreement between the ensemble members.
- Directed Exploration: During training, the robot follows an Upper Confidence Bound (UCB) strategy. It picks actions that maximize a combination of the expected reward (success) and the information gain (uncertainty). This drives the robot to "experiment" in areas where its model is most unsure.
Datasets
The researchers evaluated IDA in both simulation and the real world:
- ManiSkill2 Benchmark: A high-fidelity simulator used for tasks including grasping (Cube, YCB, and EGAD object sets), stacking cubes, and opening drawers.
- Real-World Setup: A UFACTORY xArm 6 robot equipped with a wrist-mounted RGBD camera. The robot was tasked with learning to grasp four distinct toy objects (a duck, a bowling pin, a ball, and a die) from scratch.
Key Insights and Ideas
- Active Learning vs. Big Data: The paper demonstrates that well-directed, interactive learning can outperform models pre-trained on massive synthetic datasets (like Contact-GraspNet). By interacting with the specific objects at hand, the robot learns more relevant features.
- The Power of Disagreement: Using the JSD to find where ensemble members disagree provides a precise "map of ignorance." This allows the robot to ignore empty space or already-mastered areas and focus its physical "thinking" on the edges and textures of complex objects.
- Balancing Curiosity and Success: Pure curiosity (sampling only for information) can be slow. By using UCB to include a reward term, the robot first consolidates "easy" affordances, which provides a stable foundation for exploring more difficult interactions later.
Key Results
IDA significantly outperformed standard baselines (Random, Greedy, and Where2Act) across all tested categories.
- Simulation: In the ManiSkill2 benchmark, IDA reached the highest mean success rates. Notably, on the difficult "Open Drawer" task, IDA achieved nearly 100% success, while the Greedy baseline struggled at around 73%.
- Real-World: The robot achieved a 90% grasping success rate after only a few hundred interactions. As shown in the results below, IDA learns significantly faster than random sampling or the Where2Act baseline.
- Data Efficiency: The authors found that interactive learning methods eventually outperformed pre-training on 17.7 million simulated grasps, proving that "learning by doing" in the target environment is superior to massive general pre-training.
Limitations and Future Work
- Reliance on Primitives: The current system relies on pre-defined motion planning for action primitives (e.g., "move to point X and close gripper"). It does not yet learn the low-level motor control from scratch.
- Early-Stage Noise: In the very first few interactions (under 50), IDA can sometimes perform worse than random sampling because its uncertainty estimates haven't "settled" yet.
- Long-Horizon Tasks: While IDA is excellent at discovering single-step affordances, the authors aim to extend this to hierarchical controllers or Large Language Models (LLMs) to sequence these discovered actions into complex, multi-stage plans.
Summary updated