Abstract
Domain Randomization (DR) is a cornerstone of sim-to-real transfer in robotics, yet finding the optimal distribution of simulation parameters remains a manual and often suboptimal process. We present GoFlow, a framework that leverages normalizing flows to automatically discover expressive environment distributions. GoFlow optimizes for both policy performance and distribution entropy, allowing it to capture complex, non-linear parameter dependencies that simpler methods miss. Furthermore, the learned distributions serve as "preconditions" for high-level planners, enabling robots to recognize uncertainty and trigger active perception. Our results demonstrate superior coverage in standard benchmarks and a 90% success rate in real-world contact-rich manipulation tasks.
Introduction
Training robotic policies in simulation offers safety and speed, but the "sim-to-real gap" often prevents direct deployment. Domain Randomization (DR) addresses this by varying simulation parameters (e.g., friction, mass) during training. However, standard DR faces a "Goldilocks" problem: distributions that are too narrow fail to generalize, while those that are too broad can make tasks unsolvable, leading to reinforcement learning (RL) convergence issues. Existing automated DR methods typically rely on simple distributions like Gaussians, which cannot model the complex, coupled dependencies often found in physical systems.
The GoFlow Framework
GoFlow replaces hand-tuned or simple parametric distributions with Normalizing Flows (specifically Neural Spline Flows). This allows the system to represent highly expressive, multi-modal, and non-linear distributions of simulation parameters $\xi$.
Joint Optimization and Importance Sampling
The framework optimizes a three-part objective: maximizing expected reward (performance), maximizing entropy (diversity), and minimizing the KL divergence from the previous iteration (stability). To prevent the distribution from collapsing into trivial "safe" regions, GoFlow employs importance sampling from a broad uniform distribution. This encourages the flow to expand and discover the entire "region of competence" where the task remains solvable.
Skill Sequencing and Preconditions
A unique advantage of GoFlow is that the learned distribution $p(\xi)$ acts as a map of the policy's expertise. In the real world, a robot can compare its current belief about environment parameters to this distribution. If the belief falls outside the "trusted region," a high-level planner can sequence an "inspection" action to reduce uncertainty before attempting a high-stakes manipulation.
Experimental Evaluation
We evaluated GoFlow across a variety of domains, including the IsaacLab suite (Cartpole, Ant, Quadcopter, Quadruped, Humanoid) and a real-world gear insertion task using a Franka Emika robot.
Coverage and Robustness
In simulated benchmarks, GoFlow consistently outperformed baselines such as OpenAI’s ADR and LSDR in terms of "coverage"—the proportion of the total parameter space where the policy achieves success. By modeling irregularly shaped feasible regions, GoFlow ensures the policy is robust to a wider range of environment variations.
Sim-to-Real Performance
For the contact-rich gear insertion task, GoFlow achieved a 90% success rate on real hardware. In comparison, "Full Domain Randomization" (broad but unoptimized) achieved 60%, and "No Domain Randomization" achieved only 30%. The uncertainty-aware planner correctly identified when to perform inspection steps for smaller, more difficult gears, demonstrating the utility of the learned flow as a precondition for active perception.
Conclusion
GoFlow demonstrates that expressive normalizing flows can automate the discovery of robust domain randomization distributions. By capturing the true "region of competence" of a policy, it not only improves sim-to-real transfer but also provides a mechanism for uncertainty-aware high-level planning. Future work will focus on improving the stability of flow optimization and automating threshold selection for belief-space planning.
Summary updated