On the use of simulation in robotics: Opportunities, challenges, and suggestions for moving forward

HeeSun Choi, Cindy Crump, Christian Duriez, Asher Elmquist, Gregory D. Hager, David K. Han, Frank J. Hearl, Jessica K. Hodgins, Abhinandan Jain, Frederick A. Leve, Chen Li, Franziska Meier, Dan Negruţ, Ludovic Righetti, Alberto Rodríguez, Jie Tan, Jeff TrinkleProceedings of the National Academy of Sciences 2020 · DOI

TL;DR

This perspective piece, born from a workshop of experts, argues that well-validated simulation is crucial for the safe, fast, and cost-effective development of future smart robots. It outlines the immense opportunities simulation offers, details the significant barriers to its widespread adoption, and proposes concrete steps to overcome these challenges and accelerate progress in the field.

What Problem Does This Paper Address?

The paper addresses the growing need for advanced robotics that can operate safely and effectively in dynamic, unstructured environments, often alongside humans. While physical testing of these complex robots is essential, it's often slow, expensive, and even dangerous. The authors highlight that computer simulation, when properly validated, can serve as an invaluable virtual proving ground. However, simulation in robotics faces numerous hurdles that prevent its full potential from being realized, hindering the rapid development and deployment of these next-generation robots.

Methods

This paper is a "Perspective" piece, synthesizing the collective insights from a workshop involving experts from academia, government, and industry. It doesn't present novel algorithms or experimental results in the traditional sense. Instead, it uses a structured approach to:

  • Identify Opportunities: Categorize the key areas where simulation can significantly benefit robotics development (e.g., data generation for ML, accelerating design cycles, virtual testing, HRI understanding).
  • Analyze Barriers: Detail the challenges that currently limit the broad adoption of simulation in robotics, ranging from technical issues like model complexity and speed to broader concerns like the immaturity of modeling languages and the need for multidisciplinary expertise.
  • Propose Solutions: Offer concrete recommendations and high-level observations for fostering the growth and effectiveness of simulation in robotics.

The "methods" here are essentially a structured synthesis of expert opinion and a forward-looking roadmap.

Datasets

Not discussed in the paper.

Key Insights and Ideas

  • Simulation as a "Virtual Proving Ground": The core idea is that validated simulation offers a safe, fast, and cost-effective environment for designing, testing, and verifying robots, especially those interacting with humans or operating in unpredictable conditions.
  • The "Simulation-to-Reality Gap" is a Major Hurdle: While simulation offers many advantages, the challenge of ensuring that models and learned behaviors translate effectively to the real world remains a significant barrier.
  • Multidisciplinarity is Key (and a Challenge): Building effective robotics simulation platforms requires expertise spanning computer science, mechanical engineering, physics, and even psychology (for HRI). This multidisciplinary nature is both an opportunity and a significant challenge.
  • Open-Source and Community Efforts are Crucial: The paper strongly advocates for open-source simulation platforms and community-curated libraries of validated models to democratize access, foster collaboration, and accelerate progress.
  • Human-Robot Interaction (HRI) is a Critical Frontier: Simulating human behavior and interaction with robots is essential for developing safe and effective collaborative robots, but it's also one of the most complex areas to model.
  • The Need for a "Grand Challenge" Approach: The authors suggest that organized competitions, similar to DARPA challenges, can focus research efforts, encourage cross-disciplinary collaboration, and drive the development of practical simulation tools.

Key Results

This paper doesn't present quantitative results in the traditional sense. Its "results" are the identified opportunities, challenges, and recommendations. Key takeaways include:

  • Opportunities:
    • Generating vast amounts of training data for machine learning at low cost.
  • Accelerating engineering design cycles and reducing costs.
  • Providing a safe, controlled environment for virtual testing and verification.
  • Facilitating the development of more intelligent and adaptable robots.
  • Improving the understanding of human-robot interactions (HRI).
  • Barriers:
    • Scarcity of comprehensive, validated simulation solutions requiring broad expertise.
  • Immaturity of modeling languages and robotics simulation ontologies.
  • Difficulties with model composability and scalability.
  • Simulation speed limitations, especially for complex dynamics and real-time HRI.
  • Inadequate handling of uncertainty and model calibration.
  • Slow emergence of data-driven approaches (surrogate models).
  • Challenges in determining the appropriate level of model complexity.
  • Difficulties in accurately simulating human-robot interaction.
  • Recommendations:
    • Foster open-source simulation platforms and validated model libraries.
  • Organize simulation-in-robotics grand challenge competitions.
  • Encourage research into characterizing human-robot interaction.
  • Promote simulation solutions for soft robotics.
  • Emphasize the use of emerging hardware architectures.

Limitations and Future Work

Limitations:

  • The paper acknowledges that its list of barriers and recommendations is not exhaustive.
  • It notes that the "simulation-to-reality gap" is an ongoing area of active research.
  • The authors point out that the development of robust HRI simulation models is still in its infancy, with limited understanding of psychological underpinnings.
  • The paper highlights that the argument for open-source code growing faster due to volunteers has proven largely unsubstantial in their experience, suggesting that significant expertise is required for meaningful contributions.

Future Work:

The paper implicitly and explicitly suggests numerous avenues for future work:

  • Developing more comprehensive and validated open-source simulation platforms.
  • Establishing community-curated libraries of validated models for robots, virtual worlds, and sensors.
  • Characterizing human-robot interaction more accurately, including psychological and cognitive traits.
  • Developing simulation solutions specifically for soft robotics.
  • Leveraging emerging hardware architectures (e.g., GPUs, federated approaches) to improve simulation speed and capability.
  • Investigating data-driven approaches and surrogate models to complement or replace traditional simulation.
  • Establishing principled methodologies for designing and validating data-driven models in robotics.
  • Developing better techniques for handling uncertainty in simulation.
  • Creating more sophisticated modeling languages and ontologies for robotics simulation.
  • Fostering a sustained, multidisciplinary effort to build a robust simulation-in-robotics ecosystem, potentially through broad, multi-agency initiatives.

Summary updated

Abstract

The last five years marked a surge in interest for and use of smart robots, which operate in dynamic and unstructured environments and might interact with humans. We posit that well-validated computer simulation can provide a virtual proving ground that in many cases is instrumental in understanding safely, faster, at lower costs, and more thoroughly how the robots of the future should be designed and controlled for safe operation and improved performance. Against this backdrop, we discuss how simulation can help in robotics, barriers that currently prevent its broad adoption, and potential steps that can eliminate some of these barriers. The points and recommendations made concern the following simulation-in-robotics aspects: simulation of the dynamics of the robot; simulation of the virtual world; simulation of the sensing of this virtual world; simulation of the interaction between the human and the robot; and, in less depth, simulation of the communication between robots. This Perspectives contribution summarizes the points of view that coalesced during a 2018 National Science Foundation/Department of Defense/National Institute for Standards and Technology workshop dedicated to the topic at hand. The meeting brought together participants from a range of organizations, disciplines, and application fields, with expertise at the intersection of robotics, machine learning, and physics-based simulation.
BibTeX
@inproceedings{choi2020use,
  author = {HeeSun Choi and Cindy Crump and Christian Duriez and Asher Elmquist and Gregory D. Hager and David K. Han and Frank J. Hearl and Jessica K. Hodgins and Abhinandan Jain and Frederick A. Leve and Chen Li and Franziska Meier and Dan Negruţ and Ludovic Righetti and Alberto Rodríguez and Jie Tan and Jeff Trinkle},
  title = {{On the use of simulation in robotics: Opportunities, challenges, and suggestions for moving forward}},
  booktitle = {Proceedings of the National Academy of Sciences},
  year = {2020},
  doi = {10.1073/pnas.1907856118},
}
Figure