Unsupervised Feature Learning for Manipulation with Contrastive Domain Randomization

Carmel Rabinovitz, Niko Grupen, Aviv Tamar

TL;DR

Contrastive Domain Randomization (CDR) fixes a fundamental flaw in how unsupervised contrastive learning interacts with domain randomization. By decoupling environmental noise from physical properties during training, it enables robots to learn visual features in simulation that transfer robustly to the messy, unpredictable real world without requiring any real-world labels.

What Problem Does This Paper Address?

For a robot to manipulate objects—like picking up a tool or untangling a rope—it needs to extract physical properties (like pose and shape) from camera images. Learning these features in the real world is difficult because data collection is slow and manual labeling is expensive.

Physical simulators offer a solution: they provide endless data and "free" labels. However, there is a "sim-to-real gap"—a model trained on clean, simulated images often fails when faced with real-world lighting, textures, and backgrounds. The standard fix is Domain Randomization (DR), where you randomly vary these irrelevant visual properties during training.

The authors of this paper identify a critical problem: when you combine DR with Contrastive Predictive Coding (CPC)—a popular unsupervised learning method—it doesn't actually work. Standard contrastive loss tries to maximize the information between a prediction and a target. If the background texture is the same in both, the model will "cheat" by learning to recognize the texture rather than the physics.

Methods

The researchers propose a simple but mathematically grounded fix called Contrastive Domain Randomization (CDR).

In standard CPC, you might try to predict a future frame's representation from the current one. If both frames have the same random texture, the model learns that texture. CDR breaks this link. As shown in the causal models in Figure 1, the goal is to move from a domain-dependent model to a domain-invariant one.

Causal models illustrating the difference between domain-dependent and domain-invariant representations.

The CDR Approach: 1. Take a physical state (e.g., the position of a rope). 2. Render it in two completely different random environments (different lighting, different textures). 3. Force the contrastive loss to match these two observations.

Because the only thing these two images share is the underlying physics, the neural network is forced to ignore the "noise" (textures/lighting) and focus entirely on the physical properties. The authors provide a theoretical proof (Proposition 1) showing that under mild assumptions, this objective nullifies the influence of irrelevant environmental features.

Datasets

The authors evaluated their method using the Bullet physics simulator across three distinct environments:

  • Uncontrolled Rigid Objects: A cube and a ball colliding in a frame.
  • Controlled Rigid Objects: A hammer and a banana being pushed by a robotic agent.
  • Controlled Deformable Rope: A rope being manipulated by a robotic arm.

For randomization, they used the Describable Textures Dataset (DTD), which contains thousands of real-world textures. Finally, they tested the learned representations on a real-world Franka Emika Panda robot setup (Figure 5) to prove the sim-to-real transfer.

The real-world experimental setup featuring a Panda robot arm.

Key Insights and Ideas

  • The "Shortcut" Problem: Contrastive learning is highly efficient at finding shortcuts. If a random background persists across frames, the model will use it as a unique "fingerprint" to solve the contrastive task, completely ignoring the objects it's supposed to be tracking.
  • Intervention is Key: The power of simulation isn't just more data; it's the ability to perform "interventions." By changing the environment while keeping the physics constant, we create a perfect natural experiment for the model.
  • Physics as the Invariant: In a world of changing pixels, the laws of physics are the only constant. CDR treats physics as the "signal" and visual appearance as "noise."

Key Results

The results show that CDR-trained models are significantly better at "seeing" through visual clutter:

  • Qualitative Retrieval: When shown a real-world image, CDR could retrieve simulated images with nearly identical physical configurations, even when the backgrounds were wildly different. Naive DR failed this task, often retrieving images based on color or texture instead (Figures 3 and 4).
  • Quantitative Accuracy: In predicting physical properties like object position (Euclidean distance) and shape overlap (Intersection Over Union), CDR outperformed the baseline by a wide margin, especially on "Out of Distribution" (OOD) textures.
  • Real-World Planning: The researchers used the learned features for a "visual servoing" task where the robot had to push a cube to a goal shown in an image. On the real robot, CDR achieved a final error of 7.59 cm, while the standard baseline was twice as inaccurate at 15.26 cm (Table III).
A comparison of trajectories showing the robot successfully reaching a goal using CDR representations.

Limitations and Future Work

While powerful, CDR relies on having a simulator where you can explicitly separate physical state ($x$) from environmental noise ($e$). In some complex scenarios, these might be intertwined in ways that are hard to model.

The authors suggest that future work could explore applying CDR to even more complex manipulation tasks, such as multi-object interactions or high-speed dynamics, and integrating it with different types of self-supervised objectives beyond CPC.

Summary updated

Abstract

Robotic tasks such as manipulation with visual inputs require image features that capture the physical properties of the scene, e.g., the position and configuration of objects. Recently, it has been suggested to learn such features in an unsupervised manner from simulated, self-supervised, robot interaction; the idea being that high-level physical properties are well captured by modern physical simulators, and their representation from visual inputs may transfer well to the real world. In particular, learning methods based on noise contrastive estimation have shown promising results. To robustify the simulation-to-real transfer, domain randomization (DR) was suggested for learning features that are invariant to irrelevant visual properties such as textures or lighting. In this work, however, we show that a naive application of DR to unsupervised learning based on contrastive estimation does not promote invariance, as the loss function maximizes mutual information between the features and both the relevant and irrelevant visual properties. We propose a simple modification of the contrastive loss to fix this, exploiting the fact that we can control the simulated randomization of visual properties. Our approach learns physical features that are significantly more robust to visual domain variation, as we demonstrate using both rigid and non-rigid objects.
BibTeX
@inproceedings{rabinovitz2021unsupervised,
  author = {Carmel Rabinovitz and Niko Grupen and Aviv Tamar},
  title = {{Unsupervised Feature Learning for Manipulation with Contrastive Domain Randomization}},
  booktitle = {IEEE International Conference on Robotics and Automation},
  year = {2021},
  url = {https://openreview.net/forum?id=ahhmDLVUPe},
}
Figure