TL;DR
This paper introduces a novel visual pre-training strategy for robotic manipulation that significantly improves policy robustness to visual distribution shifts. By using a Vision Transformer pre-trained with a shape-biased distillation loss, they achieve better zero-shot transfer performance and can even boost success rates by masking out attention heads that focus on irrelevant objects.
What Problem Does This Paper Address?
Robotic manipulation policies trained from raw visual input are notoriously brittle. While standard computer vision models generalize well to new environments, robotic policies often fail when faced with even minor changes in lighting, object textures, or camera viewpoints. This fragility limits their real-world applicability. The authors aim to bridge this gap by developing pre-training techniques that equip visual encoders with a better understanding of spatial relationships and object properties, leading to more robust manipulation policies.
Methods
The core of this work lies in a specific visual pre-training strategy for Vision Transformers (ViTs). Instead of standard self-supervised learning, they employ a shape-biased distillation loss (referred to as DiNo). This loss encourages the ViT to learn representations that are sensitive to object shapes.
A key observation is that when a ViT is trained with this shape bias, its attention heads can be visualized as spatial heatmaps. These heatmaps surprisingly highlight manipulation-relevant objects in the scene. The authors then propose a novel technique: masking out attention heads that focus on task-irrelevant "distractor" objects. This masking is done by zeroing out specific weights in the transformer's attention mechanism.
Datasets
- Training: The paper leverages large-scale, diverse image datasets for pre-training, though specific datasets are not detailed beyond mentioning "larger and more task-relevant datasets" and referencing Ego-4D in the context of related work.
- Evaluation:
- FrankaKitchen: Used for training and evaluating manipulation policies, specifically the sliding door task.
- Kitchen Shift benchmark: A modified version of FrankaKitchen used to evaluate robustness under various distribution shifts, including changes in lighting (darker, brighter) and object textures (wood, metal, tile) for the sliding door.
Key Insights and Ideas
- Shape bias is crucial for robust manipulation: Pre-training visual encoders with a focus on shape, rather than just general visual features, leads to significantly more robust policies that can handle visual distribution shifts.
- Attention heads as spatial feature maps: For ViTs trained with shape bias, attention heads act as emergent segmentation tools, highlighting important objects for the task. This is a surprising and useful emergent property.
- Masking distractors improves performance: By intelligently masking out attention heads that focus on irrelevant background or distractor objects, policy performance can be boosted. This suggests a way to actively refine learned representations.
- Emergent task-relevant keypoints: Even without explicit supervision, the attention maps of fine-tuned policies can highlight task-relevant keypoints in the scene, demonstrating a form of learned spatial reasoning.
Key Results
- Improved Zero-Shot Transfer: Policies trained with DiNo-pretrained ViTs outperform baselines (including those trained on larger, more task-relevant datasets like R3M and MVP) on the Kitchen Shift benchmark by a significant margin.
- Nearly 2x Performance Boost: On the sliding door task within FrankaKitchen, the proposed method achieved nearly double the average zero-shot performance compared to the next best method.
- 13% Success Rate Improvement: By masking out attention heads that attend to distractors, the authors achieved a 13% increase in maximum success rate on the sliding door task.
- Consistency across Shifts: The identified "good" attention heads remained consistent across different texture and viewpoint shifts, highlighting their robustness.
Limitations and Future Work
- Dataset Size: The paper suggests that training larger and more task-relevant datasets (like Ego-4D) with a shape-biased loss could further improve policy learning performance.
- Generalizability of Masking: While masking out distractors showed promise, further research is needed to fully explore how to leverage attention head visualizations for policy improvement. The current method relies on visual inspection to select heads, which is not fully automated.
- Automated Head Selection: Future work could focus on developing automated methods for identifying and selecting task-relevant attention heads, moving beyond manual inspection.
- Broader Applications: Exploring the applicability of this shape-biased pre-training and attention masking to a wider range of robotic manipulation tasks and environments is a natural next step.
Summary updated