The Unsurprising Effectiveness of Pre-Trained Vision Models for Control

Simone Parisi, Aravind Rajeswaran, Senthil Purushwalkam, Abhinav Gupta

Abstract

Recent years have seen the emergence of pre-trained representations as a powerful abstraction for AI applications in computer vision, natural language, and speech. However, policy learning for control is still dominated by a tabula-rasa learning paradigm, with visuo-motor policies often trained from scratch using data from deployment environments. In this context, we revisit and study the role of pre-trained visual representations for control, and in particular representations trained on large-scale computer vision datasets. Through extensive empirical evaluation in diverse control domains (Habitat, DeepMind Control, Adroit, Franka Kitchen), we isolate and study the importance of different representation training methods, data augmentations, and feature hierarchies. Overall, we find that pre-trained visual representations can be competitive or even better than ground-truth state representations to train control policies. This is in spite of using only out-of-domain data from standard vision datasets, without any in-domain data from the deployment environments.

Introduction

In the fields of Computer Vision (CV) and Natural Language Processing (NLP), the "pre-train and fine-tune" paradigm has become the gold standard. Models like ResNet, BERT, and GPT provide a foundation of general knowledge that can be adapted to specific downstream tasks with minimal additional data. In contrast, robot learning—specifically visuo-motor control—has largely remained tethered to a "tabula rasa" approach. Most reinforcement learning (RL) and imitation learning (IL) agents are trained from scratch, requiring the model to simultaneously learn visual perception and motor control from high-dimensional pixel data.

This research investigates why the pre-training revolution has not yet fully permeated robotics. Is the visual domain gap between natural images (like ImageNet) and robotic camera feeds too vast? Or have we simply not identified the right way to leverage these existing models? By systematically evaluating Pre-trained Visual Representations (PVRs), this study demonstrates that frozen, off-the-shelf vision models are not only viable but often superior to learning from scratch.

Methodology: The PVR Framework

The authors propose a streamlined three-step pipeline for developing control policies:

  1. Freeze the Representation: A pre-trained visual model (e.g., a ResNet-50 trained on ImageNet) is selected, and its weights are frozen to prevent modification during policy training.
  2. Feature Extraction: Observations from the robot’s environment (camera frames) are passed through the frozen encoder to produce a compact feature vector.
  3. Policy Training: A relatively small, shallow neural network (typically a multi-layer perceptron or LSTM) is trained using Imitation Learning (Behavioral Cloning) to map these features to actions.

This approach decouples the "learning to see" phase from the "learning to act" phase, significantly reducing the complexity of the policy optimization process.

Experimental Design and Domains

The study's strength lies in its breadth. The researchers evaluated PVRs across four distinct and challenging control domains:

  • Habitat: A photorealistic simulator for indoor navigation (ImageNav).
  • DeepMind Control (DMC) Suite: Classic physics-based locomotion tasks, such as the Cheetah and Walker.
  • Adroit: High-dimensional tasks involving a dexterous robotic hand.
  • Franka Kitchen: Complex, multi-stage manipulation tasks using a Franka Emika robot arm.

The representations were sourced from models trained on ImageNet (object-centric) and Places (scene-centric), using various objectives including supervised learning, MoCo (Self-Supervised), and CLIP (Vision-Language).

Key Insights

1. The Domain Gap is Overrated

A common assumption in robotics is that models must be trained on in-domain data to be effective. However, the results showed that pre-training on ImageNet—despite its focus on everyday objects—provided features that were just as effective as those trained on data collected directly from the robot's environment. This suggests that the fundamental visual primitives (edges, textures, spatial relationships) learned from natural images generalize remarkably well to synthetic control environments.

2. The Superiority of Self-Supervised Learning (SSL)

Models trained via Self-Supervised Learning, particularly MoCo (Momentum Contrast), consistently outperformed supervised models. The authors found that the specific data augmentations used in SSL are critical. Specifically, "crop-only" augmentation was found to be superior for control tasks. The hypothesis is that cropping forces the model to understand relative spatial displacements and local structures, which are vital for motor control, whereas color-based augmentations might discard useful lighting and shadow cues.

3. Layer Hierarchy and the "Universal" PVR

The study revealed a clear trade-off across the layers of a deep network. Early layers, which preserve high-resolution spatial information, are better suited for fine-grained locomotion tasks (DMC). Later layers, which capture high-level semantic information, are better for navigation (Habitat). To address this, the authors introduced a "Universal PVR" by concatenating features from multiple ResNet blocks (Layers 3, 4, and 5). This combined representation proved robust across all tested domains.

Results and Performance

The empirical results challenge the necessity of ground-truth state information. In the Franka Kitchen environment, a PVR-based policy achieved a 70% success rate, notably surpassing the 62% success rate of a policy trained on ground-truth simulator states. In Habitat, the MoCo-based PVR matched the performance of ground-truth features (~90% success), while an encoder trained from scratch on the same amount of data failed to exceed a 30% success rate.

Conclusion and Future Directions

This research provides a compelling argument for shifting robotics away from tabula-rasa learning. By leveraging frozen, pre-trained vision models, researchers can build more efficient and capable control policies.

While the current study focuses on Imitation Learning, the implications for Reinforcement Learning (RL) are significant. PVRs could potentially solve the sample-inefficiency bottleneck in RL by providing a stable, high-quality feature space from the start of training. Future work will need to address the transition to physical hardware and explore whether "smart" fine-tuning—selectively updating parts of the representation—can further bridge the gap between simulation and the real world.

Summary updated

Abstract

Recent years have seen the emergence of pre-trained representations as a powerful abstraction for AI applications in computer vision, natural language, and speech. However, policy learning for control is still dominated by a tabula-rasa learning paradigm, with visuo-motor policies often trained from scratch using data from deployment environments. In this context, we revisit and study the role of pre-trained visual representations for control, and in particular representations trained on large-scale computer vision datasets. Through extensive empirical evaluation in diverse control domains (Habitat, DeepMind Control, Adroit, Franka Kitchen), we isolate and study the importance of different representation training methods, data augmentations, and feature hierarchies. Overall, we find that pre-trained visual representations can be competitive or even better than ground-truth state representations to train control policies. This is in spite of using only out-of-domain data from standard vision datasets, without any in-domain data from the deployment environments. Source code and more at https://sites.google.com/view/pvr-control.
BibTeX
@inproceedings{parisi2022unsurprising,
  author = {Simone Parisi and Aravind Rajeswaran and Senthil Purushwalkam and Abhinav Gupta},
  title = {{The Unsurprising Effectiveness of Pre-Trained Vision Models for Control}},
  booktitle = {International Conference on Machine Learning},
  year = {2022},
  url = {https://openreview.net/forum?id=Q5XYliw2Hi},
}
Figure