An End-to-End System for Accomplishing Tasks with Modular Robots: Perspectives for the AI community

Gangyuan Jing, Tarik Tosun, Mark Yim, Hadas Kress-GazitIJCAI 2017 · DOI

TL;DR

This paper presents an end-to-end system that enables modular robots to autonomously select appropriate configurations and behaviors to accomplish complex, multi-part tasks. By integrating a high-level planner, a design library, a simulation tool, and actual hardware, they demonstrate a significant step towards making modular robots practical for real-world applications.

What Problem Does This Paper Address?

Modular self-reconfigurable robots (MSRRs) offer incredible flexibility by allowing a collection of identical modules to connect and form various shapes, each suited for different tasks. However, realizing this potential is hindered by a fundamental challenge: how do you automatically determine the right robot configuration (shape) and behavior (control program) for a given task? This is akin to asking a set of LEGO bricks to build themselves into the perfect tool for a specific job without human intervention. Existing approaches often struggle with this "task-to-form" mapping, especially for complex, multi-step tasks. This paper tackles this gap by proposing a unified system that bridges high-level task specifications with the physical capabilities of modular robots.

Methods

The authors introduce a four-component system designed to address the task-to-form problem for modular robots:

  1. High-Level Mission Planner: This component takes a user-defined task and, considering the robot's capabilities and environment properties, generates a sequence of actions. It leverages ideas from previous work on automatic controller synthesis for traditional robots but extends it to handle the dynamic morphology of modular robots.
  2. Design Library: This is a curated collection of robot configurations (shapes) and their associated behaviors, each labeled with specific environment and behavior properties. This library acts as a knowledge base of what the modular robot system can do.
  3. Design and Simulation Tool (VSPARC): This interactive tool allows users to design new configurations and behaviors, test them in a physics-based 3D simulator, and then add them to the design library. VSPARC also facilitates the creation of new library entries by hosting hackathons and allowing users to share their designs.
  4. Modular Robot Hardware (SMORES-EP): The system is validated on the SMORES-EP robot platform, which uses electro-permanent magnets for connection and has actuated joints for movement.

The core of the system's intelligence lies in how it uses the design library. When a task is specified, the mission planner searches the library for configurations and behaviors that match the task's requirements and environmental constraints. If multiple actions are needed, it can even orchestrate reconfigurations between different robot shapes to complete a multi-part task. Controller synthesis, a process of generating controllers from specifications, is extended to work with this modular robot framework.

Datasets

The paper does not explicitly mention traditional datasets for training. Instead, it relies on a custom-built design library populated through:

  • VSPARC: A design and simulation tool used by students and researchers.
  • Hackathons: Three hackathons were hosted to encourage the creation of new configurations and behaviors.

The library, as of the paper's writing, contained 52 designs and 97 behaviors contributed by 20 volunteers. This library serves as the primary "dataset" for the system's task execution.

Key Insights and Ideas

  • Bridging the Gap: The most significant insight is the successful integration of a high-level task planner with a library of modular robot configurations and behaviors. This end-to-end approach is crucial for making modular robots practical.
  • Reconfiguration as a Core Capability: The system doesn't just find one shape for a task; it leverages the ability of modular robots to reconfigure themselves mid-task to adapt to different sub-problems. This is demonstrated in the multi-part room-cleaning scenario.
  • Library-Driven Intelligence: The design library acts as the system's "brain," storing and retrieving the necessary robotic "skills" (configurations and behaviors) based on task requirements. The crowdsourcing approach to populate this library is a novel way to build a comprehensive knowledge base.
  • Simulation-to-Hardware Translation: The paper highlights the successful translation of behaviors designed and tested in the VSPARC simulator directly onto the physical SMORES-EP hardware, indicating the robustness of their design and control framework.

Key Results

The paper demonstrates the system's capabilities through both simulated and hardware experiments:

  • Simulated Multi-Part Task: In a simulated scenario, the system successfully used the rollingLoop configuration to complete a task involving pushing a block to form a bridge. In a more complex scenario, it orchestrated reconfigurations between backhoe, doubleDriver, and stairClimber to overcome challenges like a high button and heavy box, and to climb stairs.
  • Hardware Demonstrations:
    • Moving a Waste Bin: The system used the swerveLifter configuration to pick up and move a waste bin, showcasing its ability to handle objects and navigate complex environments.
  • Table Exploration and Cleaning: The snake7 configuration was used to ascend to a table, and then reconfigured into a modulel with differential drive to explore, push trash off the table, and spin to alert about a coffee mug.

These demonstrations showcase the system's ability to handle tasks requiring locomotion, manipulation, and adaptive reconfiguration.

Limitations and Future Work

  • Environment Characterization: The authors acknowledge that while properties can specify task requirements, a behavior labeled with a property doesn't guarantee performance if the environment differs significantly from the one it was designed for. Developing methods to automatically characterize new environments using sensor data is a key area for future work.
  • Reconfiguration Assumptions: For hardware demonstrations, they assume reconfiguration between any two configurations is possible as long as the final configuration doesn't exceed the initial module count. This simplifies the problem but might not hold true in all scenarios.
  • Sensor Limitations: The SMORES-EP modules themselves lack sensors, relying on external tracking (AprilTags) for localization and object detection. Future work could explore integrating more onboard sensing.
  • Scalability and Complexity: While the system handles multi-part tasks, scaling to extremely complex, long-horizon tasks with many reconfigurations and environmental uncertainties remains an open challenge.

Summary updated

BibTeX
@inproceedings{jing2017end,
  author = {Gangyuan Jing and Tarik Tosun and Mark Yim and Hadas Kress-Gazit},
  title = {{An End-to-End System for Accomplishing Tasks with Modular Robots: Perspectives for the AI community}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2017},
  doi = {10.24963/ijcai.2017/686},
}
Figure