ORRB - OpenAI Remote Rendering Backend

Maciek Chociej, Peter Welinder, Lilian WengarXiv 2019 · arXiv

Introduction

The "Sim2Real" transfer problem remains a significant challenge in robotics. Reinforcement learning (RL) agents trained in simulation often fail in physical environments due to the "reality gap"—the discrepancy between simulated and real-world sensory inputs. This gap is particularly pronounced in computer vision, where unpredictable lighting, textures, and optical phenomena complicate task execution.

Domain Randomization (DR) has emerged as a robust solution, involving the training of agents across a vast array of randomized simulated environments. However, traditional game engines are optimized for human-centric experiences, prioritizing low latency and high resolution. In contrast, machine learning research requires high-throughput batch rendering of low-resolution images across distributed, headless cloud environments. To address this, OpenAI developed the Remote Rendering Backend (ORRB).

System Design and Implementation

ORRB is a high-throughput, modular rendering system built on Unity3D (for visual fidelity) and MuJoCo (for physics simulation). It operates as a standalone rendering service, facilitating the massive-scale generation of synthetic data.

Key Technical Features

  • Renderer as a Service: Utilizing a gRPC interface, ORRB is language-agnostic, allowing seamless integration with Python-based ML frameworks.
  • Single Scene Multiple States (SSMS): To maximize efficiency, ORRB shares heavy assets like geometry and textures across multiple instances. It only updates lightweight randomized parameters for each state within a batch, significantly reducing overhead.
  • Modular Component Manager: The system features a "plug-and-play" architecture for randomizers. Users can attach components to randomize materials, camera jitter, lighting intensity, and post-processing effects such as grain and bloom.
  • Optimized Capture Pipeline: To prevent CPU/GPU stalls, ORRB employs a round-robin pool of render textures and bulk DMA transfers to move completed batches back to CPU memory.
The ORRB system architecture illustrates the communication between the client and multiple render server instances via gRPC.

Experimental Setup and Evaluation

The system was evaluated using the environment from OpenAI’s Dactyl project, which involves a Shadow Dexterous Hand manipulating a cube. ORRB processes MuJoCo XML scene descriptions and generates multi-channel data, including RGB images, depth maps, segmentation masks, and screen-space normal vectors.

Examples of randomized RGB, depth, normal, and segmentation channels generated by ORRB for the Shadow Dexterous Hand environment.

Performance and Scaling

Benchmarks conducted on Google Cloud Platform using NVIDIA V100 GPUs highlight ORRB's efficiency:

  • Throughput: A single V100 GPU achieved a throughput of 1,342 frames per second (FPS).
  • Scaling: Performance scales effectively with multiple GPUs, reaching 3,514 FPS on a 4-GPU configuration.
  • Saturation: Optimal performance is reached when 16 to 24 parallel render servers saturate a single V100 GPU.

A key insight from these results is that the CPU often becomes the bottleneck. Since ML tasks typically require low-resolution images (e.g., 224x224), the GPU completes rendering tasks almost instantly, leaving the CPU to handle the overhead of scene setup and rendering call management.

Conclusion and Future Work

ORRB demonstrates that prioritizing throughput over latency is essential for large-scale robotic simulation. While the system successfully bridges the reality gap for complex tasks, some limitations remain. Unity’s lack of native headless rendering support on Linux required workarounds using Xorg instances. Additionally, CPU scaling limits were observed beyond three GPUs per node due to scene setup overhead. Future work may address these bottlenecks and the minor non-determinism inherent in GPU-based rendering.

ORRB is open-source and available at github.com/openai/orrb.

Summary updated

Abstract

We present the OpenAI Remote Rendering Backend (ORRB), a system that allows fast and customizable rendering of robotics environments. It is based on the Unity3d game engine and interfaces with the MuJoCo physics simulation library. ORRB was designed with visual domain randomization in mind. It is optimized for cloud deployment and high throughput operation. We are releasing it to the public under a liberal MIT license: https://github.com/openai/orrb .
BibTeX
@misc{chociej2019orrb,
  author = {Maciek Chociej and Peter Welinder and Lilian Weng},
  title = {{ORRB - OpenAI Remote Rendering Backend}},
  year = {2019},
  eprint = {1906.11633},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1906.11633},
}
Figure