Abstract
We present a data-driven shared control algorithm that can be used to improve a human operator's control of complex dynamic machines and achieve tasks that would otherwise be challenging, or impossible, for the user on their own. Our method assumes no a priori knowledge of the system dynamics. Instead, both the dynamics and information about the user's interaction are learned from observation through the use of a Koopman operator. Using the learned model, we define an optimization problem to compute the autonomous partner's control policy. Finally, we dynamically allocate control authority to each partner based on a comparison of the user input and the autonomously generated control. We refer to this idea as model-based shared control (MbSC). We evaluate the efficacy of our approach with two human subjects studies consisting of 32 total participants. Overall, we find that model-based shared control significantly improves task and control metrics when compared to a natural learning, or user only, control paradigm.
Introduction
Shared control aims to combine human intuition with robotic precision, creating a synergistic partnership. However, most existing shared control systems rely on an a priori mathematical model of the system’s dynamics. This requirement presents significant hurdles. Many systems, such as exoskeletons or high-performance vehicles, possess nonlinear dynamics that are difficult to model analytically. Furthermore, humans interact with machines in idiosyncratic ways, making the "joint" human-machine system notoriously hard to model. Finally, models often fail to generalize, leading to a "cold start" problem where a system requires extensive data from each new user before becoming helpful.
This research introduces Model-based Shared Control (MbSC), a framework that uses data-driven Koopman operators to assist humans in controlling complex, nonlinear systems without needing a predefined model. By learning dynamics directly from observation, the system provides effective, real-time assistance that generalizes across different users.
Methodology: Model-based Shared Control (MbSC)
The MbSC framework consists of three primary components: model learning, policy generation, and control allocation.
1. Model Learning via Koopman Operators
The core innovation is the use of the Koopman operator to represent nonlinear dynamics. While the physical system is nonlinear, the Koopman operator maps these states into a higher-dimensional Hilbert space where the system's evolution becomes linear. The researchers employ Extended Dynamic Mode Decomposition (EDMD) to approximate this operator from snapshots of user interaction data.
2. Policy Generation
Once a linear-in-feature model is established, it is integrated into an optimal control framework. For nonlinear variants, the system uses Sequential Action Control (SAC), a real-time algorithm that identifies short bursts of control to maximize performance based on the learned model.
3. Control Allocation: Maxwell’s Demon
To merge human and autonomous inputs, the framework utilizes the Maxwell’s Demon Algorithm (MDA). This acts as a "lenient" filter: it permits human input only if it moves the system in a direction aligning with the autonomous "optimal" solution (i.e., within the same half-plane). Crucially, the autonomy never adds its own force; it only filters out inputs deemed counterproductive to the task goal.
Experimental Evaluation
The efficacy of MbSC was evaluated through two human-subject studies involving 32 participants. The experimental task utilized a modified version of the OpenAI Gym Lunar Lander environment. This simulation is notoriously difficult for novices due to its unstable, low-gravity dynamics and the need to manage multiple thrusters simultaneously.
The studies compared "User Only" control against three types of MbSC models:
- Individual: Learned from the current user's own data.
- General: Learned from three other novice users.
- Expert: Learned from a highly experienced pilot.
Results and Discussion
The results demonstrate that MbSC significantly enhances performance in complex control tasks.
Performance and Success Rates
In the nonlinear study, users under MbSC achieved success rates between 60% and 80%, a dramatic improvement over the 10-20% success rate in the "User Only" condition. While a linear variant of the model showed positive trends, only the nonlinear variant provided a statistically significant boost, highlighting the necessity of capturing nonlinearities for complex tasks.
Generalization and Data Efficiency
A key finding was that "General" and "Expert" models performed as well as "Individual" models. This suggests that the Koopman operator successfully captures the fundamental physics of the human-machine interaction, eliminating the need for user-specific training. Furthermore, the system demonstrated extreme data efficiency; in online learning trials, the system became helpful after only five trials (approximately 90 seconds of data), far outpacing typical deep reinforcement learning approaches.
Agreement and Ergodicity
Users "agreed" with the autonomous controller—meaning their input passed the MDA filter—approximately 65-70% of the time. Additionally, using ergodicity as a metric, the authors showed that MbSC helps users spend more time in desirable states near the goal, even in trials that did not result in a successful landing.
Conclusion
MbSC provides a robust, data-driven framework for shared control that bypasses the need for analytical modeling. By leveraging Koopman operators, the system achieves high performance, rapid learning, and excellent generalization across users. Future work will explore haptic feedback to guide users toward optimal control planes and automated methods for basis function selection to further enhance the system's adaptability.
Summary updated