Abstract
Transferring motion instructions from a user enables robots to perform new and unforeseen operations. Robot collectives, in particular, offer greater adaptability to changing tasks and environments. However, effectively transferring motion instructions becomes challenging as the collective’s shape and size evolve. These changes often require additional system constraints to maintain robust motion control, which typically depends on pre-programmed knowledge of new tasks, ultimately limiting the collective’s adaptability. To overcome the above challenges, we propose a physical and computational platform for user-guided control of self-reconfigurable modular robots. This platform consists of an optimization scheme for online processing of user commands, which prevents any modular robot actions that would violate system or environment constraints. The second component consists of Joint-space Joysticks, which match the robot’s morphology, enabling the user to control diverse and dynamically changing modular robot structures through direct physical interaction. We present a platform that enables users to safely control modular, shape-changing robots through a physical interface. We demonstrate the platform’s efficacy and generalizability across a diverse set of modular robot morphologies using two independent robotic systems — Mori3 and Roombots — performing a range of tasks including pick-and-place, human assistance, legged locomotion, and workspace expansion. The combination of modular tangible interfaces and constrained optimization control enables users to operate shape-changing robots by matching their morphology and ensuring safety, allowing adaptation to different environments and application areas.
Introduction
Modular robots (MRs) are the "Swiss Army knives" of robotics, capable of rearranging themselves into grippers, tripods, or long arms to suit specific tasks. However, this versatility introduces significant control challenges. Standard interfaces, such as Xbox controllers, lack the necessary degrees of freedom to manage complex, multi-jointed configurations. Furthermore, as a robot changes shape, its physical limits—such as stability boundaries and joint torque capacities—evolve dynamically.
There is a massive gap between a user’s high-level intent and the low-level physics required to keep a shape-shifting robot from damaging itself or its environment. Existing methods like remote controllers or motion capture often struggle to bridge this gap effectively while maintaining safety.
System Architecture: JoJo and QP Optimization
The proposed platform addresses these challenges through a two-part solution: a physical interface that mirrors the robot's morphology and a computational "safety filter."
Joint-space Joysticks (JoJo)
The JoJo is a low-cost, unactuated physical replica of the robot's modules. It contains Hall-effect sensors to track joint angles but lacks motors. A user physically manipulates the JoJo, and its configuration is sent wirelessly to the actual robot. Because the JoJo matches the robot's morphology, the control is "joint-to-joint" and highly intuitive, eliminating the need for users to learn complex button mappings.
Quadratic Programming (QP) Optimization
The QP framework acts as the system's "brain," sitting between the user and the robot. It treats the user's input as a suggestion rather than a direct command. Every few milliseconds, the system solves a Quadratic Programming problem to find the robot movement that stays as close as possible to the user's pose while strictly obeying a set of "hard" constraints. These include:
- Static Constraints: Joint limits, velocity limits, and torque limits.
- Dynamic Constraints: Self-collision avoidance (using convex hulls), Center of Mass (CoM) stability for walking robots, and connector strength limits.
Experimental Validation and Results
The platform was validated using two distinct modular systems: Mori3 (triangular modules) and Roombots (cubic/lattice modules). Key results across diverse tasks include:
- Safety Filtering: In human-assistance scenarios, the QP layer successfully maintained a safety buffer, preventing the robot from colliding with a person even when the user's JoJo input suggested a dangerous move.
- Stability and Torque Management: For legged locomotion, the system automatically restricted movements that would have caused the robot's Center of Mass to leave its support polygon. In manipulation tasks, it successfully overrode commands that would have exceeded base joint torque limits.
- Tracking Accuracy: Within safe operating zones, the system demonstrated high precision, with a Mean Absolute Error of only 0.16 ± 4.47 degrees.
- Dynamic Reconfiguration: The controller successfully handled "hot-swapping" of modules. When new modules docked, the QP framework instantly updated its kinematic model and constraints without requiring a system reset.
Discussion and Future Outlook
The integration of modular tangible interfaces with constrained optimization enables safe, intuitive operation of shape-changing robots. While highly effective for small-scale collectives (4-6 modules), future work is needed to address the physical scalability of the JoJo interface for larger swarms. Additionally, reducing communication latency (currently 0.5 to 2.3 seconds) will be essential for high-speed reactive tasks. Moving forward, the researchers aim to explore "shared autonomy," where the robot autonomously manages low-level task stability while the user focuses on high-level directional intent.
Summary updated