TL;DR
This paper introduces an end-to-end imitation learning framework that simultaneously discovers grasping affordances and learns manipulation policies directly from human demonstrations. By framing affordances as latent visual attention maps and optimizing them via a novel coupled triplet loss, the robot learns where and how to grasp complex objects without requiring tedious, pixel-level ground-truth affordance labels.
What Problem Does This Paper Address?
When humans look at a mug, we don't just see a geometric shape; we implicitly see its affordances—the potential actions we can perform on it based on its features (e.g., grasping it by the handle to pour, or by the body to move it).
In robotics, learning these object-action-effect relationships from demonstrations typically happens in a decoupled, two-step process:
- An affordance predictor explicitly estimates grasping configurations (like gripper approach angles or preshapes).
- A separate, classic motion planner uses these configurations to execute a trajectory.
This decoupled approach is often brittle, engineered, and heavily reliant on labor-intensive ground-truth affordance labels. The authors of this paper tackle a core challenge: How can we bridge the gap between affordance discovery and policy learning within a single, end-to-end deep neural network?
Methods
The authors propose an end-to-end contrastive imitation learning framework consisting of two main components: a Siamese Encoder and a Trajectory Decoder.
[Demonstration Trajectory]
│
├──► Interaction Segment ──► LSTM Encoder ──────┐
│ ▼ [Coupled Triplet Loss]
└──► Step-by-Step Obs ──► Trajectory Encoder ┴► [Affordance Cue / Attention]
│
▼
Trajectory Decoder ──► Predicted Action
1. Siamese Encoder & Interaction Segments
To learn clean representations of affordances, the network extracts the interaction segment of a trajectory—the specific window of state-action pairs where the gripper actually manipulates the object. An LSTM encodes this low-dimensional interaction data into an affordance embedding ($Z^A$).
Concurrently, a two-branch trajectory encoder processes high-dimensional step-by-step observations (depth images, current state, and previous action) to generate an observation embedding ($z^{o,A}_t$). Crucially, this branch outputs a latent, channel-wise attention map—the affordance cue.
2. Coupled Triplet Loss
Standard contrastive learning frameworks often fail here because the network might exploit affordance-irrelevant shortcuts (like background context or initial positions) to distinguish trajectories. To prevent this, the authors introduce a coupled triplet loss. This loss forces the high-dimensional observation embeddings to be guided by the spatial relationships of the cleaner, low-dimensional affordance embeddings.
3. Trajectory Decoder
The trajectory decoder acts as the policy network. It multiplies the discovered affordance cue (attention map) with the visual feature maps, concatenates the robot's current state and contrastive embedding, and passes them through convolutional and fully connected layers to output the next action. The entire architecture is trained jointly using a combination of the coupled triplet loss and a standard behavior cloning loss.
Datasets
The framework was evaluated in a simulated environment using the PyBullet simulator and a Franka Panda robot arm.
- Objects: 24 distinct mug models featuring diverse geometric structures, sizes, and handle shapes.
- Demonstrations: 27 trajectories (each 8 steps long) collected via human teleoperation. Trajectories were categorized into three affordable grasp types:
body-grasp,handle-left-right-grasp, andhandle-front-back-grasp.
Key Insights and Ideas
- Attention as an Implicit Affordance Proxy: Instead of forcing a network to explicitly output bounding boxes or gripper angles, visual attention maps can naturally serve as a reliable latent feature for policy execution.
- The Danger of Naive Contrastive Learning: One of the most surprising findings was that applying a standard triplet loss directly to raw visual observations actually degraded performance compared to not using contrastive learning at all. Without guidance, the network latched onto irrelevant environmental features, misguiding the policy.
- Low-Dim Guiding High-Dim: By coupling the visual encoder's loss to the low-dimensional interaction segment encoder, the framework successfully anchors visual attention to the physical reality of the grasp.
Key Results
The models were evaluated on their ability to successfully grasp and lift randomly sampled mugs by 5 centimeters.
| Model | Grasping Success Rate |
|---|---|
| Ours (Full Model): Siamese + Coupled Triplet Loss | 65% |
| Ours (Ablation): Siamese + Normal Triplet Loss | 35% |
| Ours (Ablation): Without Contrastive Learning | 45% |
| Baseline (Standard Behavior Cloning) | 25% |
The full model significantly outperformed the baseline and ablation variants. Qualitatively, the generated attention maps clearly highlighted the handles for handle-graspable mugs and the bodies for mugs that required a body grasp, validating that the network truly discovered meaningful affordance cues.
Limitations and Future Work
While promising, the authors note a few limitations that pave the way for future research:
- Single-Level Interactions: The current setup only handles direct interactions between a gripper and a single object. Complex tasks involving tool use (e.g., using a hook to pull a mug closer) would require multi-level interaction modeling.
- Fixed High-Level Tasks: The affordance cues discovered were tied to a single task ("picking-to-pour-water"). If the high-level goal changes to something like "pick-and-relocate", the ideal affordance cue changes. Future iterations could integrate this method into a hierarchical imitation learning framework to condition attention on varying high-level goals.
Summary updated