Data-driven Koopman Operators for Model-based Shared Control of Human-Machine Systems

Alexander Broad, Ian Abraham, Todd D. Murphey, Brenna Argall

Abstract

We present a data-driven shared control algorithm that can be used to improve a human operator's control of complex dynamic machines and achieve tasks that would otherwise be challenging, or impossible, for the user on their own. Our method assumes no a priori knowledge of the system dynamics. Instead, both the dynamics and information about the user's interaction are learned from observation through the use of a Koopman operator. Using the learned model, we define an optimization problem to compute the autonomous partner's control policy. Finally, we dynamically allocate control authority to each partner based on a comparison of the user input and the autonomously generated control. We refer to this idea as model-based shared control (MbSC). We evaluate the efficacy of our approach with two human subjects studies consisting of 32 total participants. Overall, we find that model-based shared control significantly improves task and control metrics when compared to a natural learning, or user only, control paradigm.

Introduction

Shared control aims to combine human intuition with robotic precision, creating a synergistic partnership. However, most existing shared control systems rely on an a priori mathematical model of the system’s dynamics. This requirement presents significant hurdles. Many systems, such as exoskeletons or high-performance vehicles, possess nonlinear dynamics that are difficult to model analytically. Furthermore, humans interact with machines in idiosyncratic ways, making the "joint" human-machine system notoriously hard to model. Finally, models often fail to generalize, leading to a "cold start" problem where a system requires extensive data from each new user before becoming helpful.

This research introduces Model-based Shared Control (MbSC), a framework that uses data-driven Koopman operators to assist humans in controlling complex, nonlinear systems without needing a predefined model. By learning dynamics directly from observation, the system provides effective, real-time assistance that generalizes across different users.

Methodology: Model-based Shared Control (MbSC)

The MbSC framework consists of three primary components: model learning, policy generation, and control allocation.

The MbSC pipeline: (a) Model Learning via Koopman operators, (b) Policy Generation using optimal control, and (c) Shared Control via the Maxwell’s Demon Algorithm.

1. Model Learning via Koopman Operators

The core innovation is the use of the Koopman operator to represent nonlinear dynamics. While the physical system is nonlinear, the Koopman operator maps these states into a higher-dimensional Hilbert space where the system's evolution becomes linear. The researchers employ Extended Dynamic Mode Decomposition (EDMD) to approximate this operator from snapshots of user interaction data.

2. Policy Generation

Once a linear-in-feature model is established, it is integrated into an optimal control framework. For nonlinear variants, the system uses Sequential Action Control (SAC), a real-time algorithm that identifies short bursts of control to maximize performance based on the learned model.

3. Control Allocation: Maxwell’s Demon

To merge human and autonomous inputs, the framework utilizes the Maxwell’s Demon Algorithm (MDA). This acts as a "lenient" filter: it permits human input only if it moves the system in a direction aligning with the autonomous "optimal" solution (i.e., within the same half-plane). Crucially, the autonomy never adds its own force; it only filters out inputs deemed counterproductive to the task goal.

Experimental Evaluation

The efficacy of MbSC was evaluated through two human-subject studies involving 32 participants. The experimental task utilized a modified version of the OpenAI Gym Lunar Lander environment. This simulation is notoriously difficult for novices due to its unstable, low-gravity dynamics and the need to manage multiple thrusters simultaneously.

The modified Lunar Lander environment used for human-subject studies, showing the lander, goal, and coordinate system.

The studies compared "User Only" control against three types of MbSC models:

  • Individual: Learned from the current user's own data.
  • General: Learned from three other novice users.
  • Expert: Learned from a highly experienced pilot.

Results and Discussion

The results demonstrate that MbSC significantly enhances performance in complex control tasks.

Performance and Success Rates

In the nonlinear study, users under MbSC achieved success rates between 60% and 80%, a dramatic improvement over the 10-20% success rate in the "User Only" condition. While a linear variant of the model showed positive trends, only the nonlinear variant provided a statistically significant boost, highlighting the necessity of capturing nonlinearities for complex tasks.

Generalization and Data Efficiency

A key finding was that "General" and "Expert" models performed as well as "Individual" models. This suggests that the Koopman operator successfully captures the fundamental physics of the human-machine interaction, eliminating the need for user-specific training. Furthermore, the system demonstrated extreme data efficiency; in online learning trials, the system became helpful after only five trials (approximately 90 seconds of data), far outpacing typical deep reinforcement learning approaches.

Agreement and Ergodicity

Users "agreed" with the autonomous controller—meaning their input passed the MDA filter—approximately 65-70% of the time. Additionally, using ergodicity as a metric, the authors showed that MbSC helps users spend more time in desirable states near the goal, even in trials that did not result in a successful landing.

Conclusion

MbSC provides a robust, data-driven framework for shared control that bypasses the need for analytical modeling. By leveraging Koopman operators, the system achieves high performance, rapid learning, and excellent generalization across users. Future work will explore haptic feedback to guide users toward optimal control planes and automated methods for basis function selection to further enhance the system's adaptability.

Summary updated

Abstract

We present a data-driven shared control algorithm that can be used to improve a human operator's control of complex dynamic machines and achieve tasks that would otherwise be challenging, or impossible, for the user on their own. Our method assumes no a priori knowledge of the system dynamics. Instead, both the dynamics and information about the user's interaction are learned from observation through the use of a Koopman operator. Using the learned model, we define an optimization problem to compute the autonomous partner's control policy. Finally, we dynamically allocate control authority to each partner based on a comparison of the user input and the autonomously generated control. We refer to this idea as model-based shared control (MbSC). We evaluate the efficacy of our approach with two human subjects studies consisting of 32 total participants (16 subjects in each study). The first study imposes a linear constraint on the modeling and autonomous policy generation algorithms. The second study explores the more general, nonlinear variant. Overall, we find that model-based shared control significantly improves task and control metrics when compared to a natural learning, or user only, control paradigm. Our experiments suggest that models learned via the Koopman operator generalize across users, indicating that it is not necessary to collect data from each individual user before providing assistance with MbSC. We also demonstrate the data-efficiency of MbSC and consequently, it's usefulness in online learning paradigms. Finally, we find that the nonlinear variant has a greater impact on a user's ability to successfully achieve a defined task than the linear variant.
BibTeX
@article{broad2020data,
  author = {Alexander Broad and Ian Abraham and Todd D. Murphey and Brenna Argall},
  title = {{Data-driven Koopman Operators for Model-based Shared Control of Human-Machine Systems}},
  journal = {Int. J. Robotics Res.},
  year = {2020},
  url = {https://openreview.net/forum?id=AkuWfhGj9V},
}
Figure