Trust Recognition in Human-Robot Cooperation Using EEG

Caiyue Xu, Changming Zhang, Yanmin Zhou, Zhipeng Wang, Ping Lu, Bin He

TL;DR

This paper introduces a novel EEG-based method for recognizing human trust during human-robot cooperation. By employing a 3D spatial representation of EEG signals and a Vision Transformer architecture, the approach achieves promising accuracy in predicting trust levels, outperforming existing methods and paving the way for more intuitive and adaptive human-robot interactions.

What Problem Does This Paper Address?

As robots become more integrated into our daily lives, effective human-robot cooperation is paramount. A key factor for successful collaboration is trust: humans need to trust robots to perform tasks reliably, and robots need to understand human trust levels to make appropriate decisions. However, accurately and in real-time recognizing human trust, especially within dynamic cooperation scenarios, remains a significant challenge. Existing methods often rely on subjective self-reports (which interrupt interaction) or behavioral cues (which can be scenario-specific and hard to generalize). This paper aims to bridge this gap by developing a generalized, real-time EEG-based approach to recognize human trust during human-robot cooperation.

Methods

The core of this work lies in two main innovations:

  • Human-Robot Cooperation Game for Trust Stimulation: The researchers designed a human-robot cooperation game using the "Overcooked-AI" environment. This game was specifically crafted to elicit varying levels of human trust by manipulating robot ability (high, medium, low) and task complexity (five different kitchen layouts). This controlled environment allowed for the systematic collection of EEG data corresponding to different trust states.
  • EEG Vision Transformer with 3D Spatial Representation: To process the EEG data, the authors propose a novel model. First, they extract Differential Entropy (DE) features from the EEG signals across different frequency bands. Crucially, instead of treating EEG as sequential data, they transform it into a 3D spatial representation. This representation maps the electrode locations onto a mesh grid, capturing the topological relationships between electrodes and thus encoding spatial information. This 3D representation is then fed into a Vision Transformer (ViT) architecture, which is adept at processing image-like data and leveraging spatial attention mechanisms to identify patterns indicative of trust.

Datasets

The study introduces a new benchmark dataset called EEGTrust. This dataset was constructed from the experiments conducted with 16 participants playing the Overcooked-AI game. It contains EEG data collected during these cooperative tasks, along with subjective trust ratings obtained after each trial. This dataset is made publicly available.

Key Insights and Ideas

  • Cooperation is Key for Trust Elicitation: Unlike previous studies that often focused on humans observing robots, this work demonstrates that a cooperative task scenario is effective in stimulating and measuring varying levels of human trust.
  • Spatial Information Matters in EEG for Trust: The paper strongly suggests that the spatial arrangement of EEG electrodes is crucial for accurate trust recognition. Simply treating EEG as sequential data misses important information that can be captured by a 3D spatial representation.
  • Vision Transformers Adapt Well to EEG: The Vision Transformer architecture, originally designed for image analysis, proves effective in processing the spatially represented EEG data, highlighting its potential for analyzing complex neural signals.
  • Trust Variance: The study observed that trust states tend to have greater variance than distrust states, suggesting that humans might be more nuanced and less certain when building trust compared to when they are distrustful.

Key Results

The proposed EEG Vision Transformer model achieved significant results:

  • Slice-wise Cross-Validation: An accuracy of 74.99% and an F1 score of 73.41%.
  • Trial-wise Cross-Validation: An accuracy of 62.00% and an F1 score of 62.59%.

These results consistently outperformed traditional machine learning models (NB, KNN, SVM) and a baseline CNN model, particularly in the more challenging trial-wise cross-validation, which better reflects real-world deployment scenarios. An ablation study confirmed that the 3D spatial representation significantly improved performance, yielding a 13.24% accuracy increase in slice-wise validation and a 6.92% increase in trial-wise validation compared to a model without spatial representation.

Limitations and Future Work

The authors acknowledge a key limitation: performance variability across subjects. This suggests that while the model is effective, its adaptability to individual differences needs improvement.

Future work directions include:

  • Enhancing Adaptability: Developing methods to improve the model's robustness to individual variations in EEG signals.
  • Real-time Implementation: Focusing on creating a software architecture for real-time trust recognition.
  • Neuroscience Advancements: Investigating the relationship between EEG signals from different brain regions to refine trust measurement and contribute to a deeper understanding of trust from a neuroscience perspective.

Summary updated

Abstract

Collaboration between humans and robots is becoming increasingly crucial in our daily life. In order to accomplish efficient cooperation, trust recognition is vital, empowering robots to predict human behaviors and make trust-aware decisions. Consequently, there is an urgent need for a generalized approach to recognize human-robot trust. This study addresses this need by introducing an EEG-based method for trust recognition during human-robot cooperation. A human-robot cooperation game scenario is used to stimulate various human trust levels when working with robots. To enhance recognition performance, the study proposes an EEG Vision Transformer model coupled with a 3-D spatial representation to capture the spatial information of EEG, taking into account the topological relationship among electrodes. To validate this approach, a public EEG-based human trust dataset called EEGTrust is constructed. Experimental results indicate the effectiveness of the proposed approach, achieving an accuracy of 74.99% in slice-wise cross-validation and 62.00% in trial-wise cross-validation. This outperforms baseline models in both recognition accuracy and generalization. Furthermore, an ablation study demonstrates a significant improvement in trust recognition performance of the spatial representation. The source code and EEGTrust dataset are available at https://github.com/CaiyueXu/EEGTrust.
BibTeX
@inproceedings{xu2024trust,
  author = {Caiyue Xu and Changming Zhang and Yanmin Zhou and Zhipeng Wang and Ping Lu and Bin He},
  title = {{Trust Recognition in Human-Robot Cooperation Using EEG}},
  booktitle = {IEEE International Conference on Robotics and Automation},
  year = {2024},
  url = {https://openreview.net/forum?id=iD4VQrpAbS},
}
Figure