An embodiment paradigm in evaluation of human-in-the-loop control

Jakob Fröhner, Philipp Beckerle, Satoshi Endo, Sandra HircheIFAC-PapersOnLine 2019 · DOI

TL;DR

This paper proposes using the psychological concept of "embodiment" – the feeling of an external object being part of one's own body – as a novel metric to evaluate the quality of human-in-the-loop control systems. They found that predictable semi-autonomous assistance significantly enhances embodiment, suggesting it's a valuable user-centric measure for designing better human-robot interactions.

What Problem Does This Paper Address?

As robots and assistive technologies become more integrated into our lives (think teleoperation, mobility aids, prosthetics), ensuring a positive user experience is crucial. While semi-autonomous control aims to improve task performance by taking over certain sub-tasks, it can paradoxically reduce a user's sense of control, leading to frustration and a negative experience. Current evaluation methods primarily focus on objective task performance (speed, accuracy, etc.), but they don't capture this crucial subjective aspect of how well a user feels integrated with the system. This paper addresses this gap by exploring how the psychological construct of embodiment can serve as a quantitative measure for the quality of human-machine control.

Methods

The researchers developed a virtual reality (VR) setup where participants performed a 2D reaching task. They used a haptic device to simulate a virtual arm, which was assisted by a semi-autonomous controller. The core of their methodology involved manipulating two key factors:

  • Synchronicity of Visual Feedback: The visual feedback of the virtual arm's movement was either perfectly synchronized with the user's physical input or delayed by 500ms.
  • Predictability of Assistance: The semi-autonomous controller either behaved predictably (a consistent 50/50 split of control between human and robot) or unpredictably (a randomly drifting control allocation).

After performing the task under these four conditions (2x2 factorial design), participants completed questionnaires designed to measure their sense of embodiment (specifically, ownership, location, and agency) and presence within the virtual environment. Objective performance metrics like RMS jerk, mean scalar velocity, and compensation force were also recorded.

Datasets

Not discussed in the paper. The study used a custom-built experimental setup and a virtual environment.

Key Insights and Ideas

  • Embodiment as a User-Centric Metric: The paper champions embodiment as a powerful, user-centric way to evaluate control systems, moving beyond purely performance-based metrics.
  • Predictability is Key: The predictability of the semi-autonomous controller had a significant impact on embodiment. Users felt more "connected" to the virtual arm when the assistance was predictable.
  • Synchronicity Matters for Agency and Ownership: Delayed visual feedback (asynchronous conditions) negatively impacted the sense of agency and showed a trend towards affecting ownership, highlighting the importance of real-time sensory-motor feedback.
  • Embodiment Subscales Reveal Nuances: Breaking down embodiment into ownership, location, and agency provided a more detailed understanding of how different control aspects affect the user experience. Predictability strongly influenced location, while agency and ownership showed trends.
  • Presence Correlates with Performance: The study found that higher presence scores were associated with better task performance (lower jerk, higher velocity), aligning with previous research.

Key Results

  • Subjective Measures:
    • The overall embodiment score was significantly higher with predictable control (0.76±0.84) compared to unpredictable control (0.25±1.13).
  • Synchronous visual feedback also yielded higher embodiment scores (0.96±0.97) than asynchronous feedback (-0.45±1.00).
  • These effects were consistent across subscales of embodiment and aspects of presence.
  • Objective Measures:
    • RMS jerk was significantly higher with unpredictable control and asynchronous feedback.
  • Mean scalar velocity was significantly higher with synchronous feedback.
  • Compensation force was not significantly affected by predictability but was lower in synchronous conditions.

Limitations and Future Work

  • Small Sample Size: The study involved only 8 participants, which limits the generalizability of the findings.
  • Generalizability of Embodiment: While embodiment proved effective, further studies with more participants are needed to fully understand how its subscales are influenced by varying levels of predictability and autonomy in assistive control.
  • Task Specificity: The findings are based on a specific 2D reaching task in VR. Future work could explore how these results translate to more complex, real-world tasks and different types of human-robot interaction.
  • Exploring Autonomy Levels: The paper focused on shared control. Investigating the impact of different degrees of autonomy on embodiment would be a valuable next step.

Summary updated

Abstract

This study introduces a novel approach for evaluating the quality of human-in-the-loop control using the psychological construct of embodiment in a haptic human-machine interaction task. Despite the fact that various forms of assistive control have been introduced, these methods mainly design semi-autonomous control to improve task-specific interaction performance. From a user perspective, however, the introduction of semi-autonomous control reduces controllability of the system, which could lead to negative user experience. Psychological research suggests sensory-motor factors dynamically modulate cognition of an external entity belonging to one’s own body, i.e., embodiment. In our paradigm, we apply methods for evaluating embodiment in a virtual reality (VR) environment where the human users perform a reaching task with semi-autonomous haptic assistance to measure the degree to which the embodiment is effected by the quality of semi-autonomy Our results with 8 participants show good persistence of subjective embodiment and subcomponents of presence within VR environment when a predictable assistive control is introduced while unpredictable assistance hindered the subjective embodiment. Results indicate embodiment can be exploited as a quantitative evaluation method of semi-autonomous controllers from a user-centric perspective.
BibTeX
@inproceedings{frohner2019embodiment,
  author = {Jakob Fröhner and Philipp Beckerle and Satoshi Endo and Sandra Hirche},
  title = {{An embodiment paradigm in evaluation of human-in-the-loop control}},
  booktitle = {IFAC-PapersOnLine},
  year = {2019},
  doi = {10.1016/j.ifacol.2019.01.036},
}
Figure