Design of a EEG-based Drone Swarm Control System using Endogenous BCI Paradigms

Dae-Hyeok Lee, Hyung-Ju Ahn, Ji-Hoon Jeong, Seong-Whan LeearXiv 2020 · OpenReview

TL;DR

This paper presents a novel system for controlling drone swarms using endogenous Brain-Computer Interface (BCI) paradigms (motor, visual, and speech imagery). By leveraging these "thought-based" commands, the system aims to increase the degrees of freedom for drone swarm control, offering more intuitive and flexible operation for complex tasks.

What Problem Does This Paper Address?

Controlling multiple drones simultaneously, known as drone swarms, offers significant advantages in various applications, from military operations to disaster response. However, current control methods often lack the intuitive and flexible control needed for complex, multi-drone missions. Traditional BCI systems for drone control have often relied on exogenous paradigms, which require external stimuli and can be less natural for the user. This paper addresses the need for a more intuitive and higher-degree-of-freedom control system for drone swarms by exploring the potential of endogenous BCI paradigms, which allow users to control systems based on their internal thoughts and intentions without external cues.

Methods

The core of this research lies in designing and implementing three specific endogenous BCI paradigms:

  • Motor Imagery (MI): Subjects imagined movements of their left hand, right hand, both hands, or feet. These were mapped to drone swarm commands for left/right direction, high/low altitude, respectively.
  • Visual Imagery (VI): Subjects visualized different swarm behaviors, specifically "fall-in" (increasing cohesion), "spread-out" (decreasing cohesion), and "split" (dividing into two groups).
  • Speech Imagery (SI): Subjects imagined saying specific commands: "Go," "Stop," "Follow-me," and "Return."

EEG signals were acquired from five healthy subjects performing these imagined tasks. The acquired EEG data underwent preprocessing, including down-sampling, bandpass filtering, and independent component analysis (ICA) for artifact removal. For feature extraction, Common Spatial Pattern (CSP) was employed, and classification was performed using Linear Discriminant Analysis (LDA). The system architecture integrates EEG acquisition, signal processing, and a drone swarm control module via ROS. Performance was evaluated using 5-fold cross-validation.

Datasets

The paper does not explicitly refer to pre-existing, publicly available datasets. Instead, it describes the creation of a new dataset by conducting experiments with five healthy subjects performing the designed MI, VI, and SI tasks. This dataset was used for training and evaluating the classification algorithms.

Key Insights and Ideas

  • Endogenous BCI for Drone Swarms: The study demonstrates the feasibility of using endogenous BCI paradigms for drone swarm control, moving beyond stimulus-dependent methods. This opens doors for more natural and intuitive human-swarm interaction.
  • Increased Degrees of Freedom: By mapping distinct imagined tasks to specific drone swarm commands, the researchers successfully increased the potential degrees of freedom for control, enabling more nuanced maneuvers and mission objectives.
  • Task-Specific Paradigm Design: The authors tailored the endogenous paradigms (MI, VI, SI) to specific aspects of drone swarm control (direction, formation, mission commands), highlighting the importance of context-specific BCI design.
  • Feasibility Confirmation: Despite using basic machine learning algorithms, the results show accuracies significantly above chance level, confirming the potential of this approach.

Key Results

The study reports the following grand-averaged classification accuracies for the endogenous BCI paradigms:

  • Motor Imagery (MI): 51.1% (± 8.02)
  • Visual Imagery (VI): 53.2% (± 3.11)
  • Speech Imagery (SI): 41.9% (± 6.09)

These accuracies were consistently higher than the chance-level accuracies for the respective number of classes (25.0% for 4-class MI/SI, 33.3% for 3-class VI). The paper also notes that while some subjects performed poorly, the overall results suggest the acquired EEG data was of high quality.

Limitations and Future Work

Limitations:

  • Basic Machine Learning: The study utilized basic machine learning algorithms (LDA), and the authors acknowledge that more advanced techniques could potentially improve performance.
  • Subject-Specific Performance: Some subjects exhibited lower performance, which the authors attribute to the difficulty of performing the imagined tasks. This highlights the need for robust subject-independent or adaptive BCI systems.
  • Limited Degrees of Freedom: While the study aimed to increase degrees of freedom, the current implementation still has limitations compared to full manual control.

Future Work:

  • Deep Learning Models: The authors plan to design deep learning network models that utilize EEG signals from endogenous paradigms to achieve higher performance.
  • Robust Real-World Application: Developing systems that can robustly adapt to real-world environments and allow for seamless switching between different BCI paradigms is a key future direction.
  • Intuitive Instruction: Further research is needed to make the imagined tasks more intuitive and aligned with real-world drone swarm control scenarios.

Summary updated

Abstract

Non-invasive brain-computer interface (BCI) has been developed for understanding users' intentions by using electroencephalogram (EEG) signals. With the recent development of artificial intelligence, there have been many developments in the drone control system. BCI characteristic that can reflect the users' intentions led to the BCI-based drone control system. When using drone swarm, we can have more advantages, such as mission diversity, than using a single drone. In particular, BCI-based drone swarm control could provide many advantages to various industries such as military service or industry disaster. BCI Paradigms consist of the exogenous and endogenous paradigms. The endogenous paradigms can operate with the users' intentions independently of any stimulus. In this study, we designed endogenous paradigms (i.e., motor imagery (MI), visual imagery (VI), and speech imagery (SI)) specialized in drone swarm control, and EEG-based various task classifications related to drone swarm control were conducted. Five subjects participated in the experiment and the performance was evaluated using the basic machine learning algorithm. The grand-averaged accuracies were 51.1%, 53.2%, and 41.9% in MI, VI, and SI, respectively. Hence, we confirmed the feasibility of increasing the degree of freedom for drone swarm control using various endogenous paradigms.
BibTeX
@misc{lee2020design,
  author = {Dae-Hyeok Lee and Hyung-Ju Ahn and Ji-Hoon Jeong and Seong-Whan Lee},
  title = {{Design of a EEG-based Drone Swarm Control System using Endogenous BCI Paradigms}},
  year = {2020},
  eprint = {2012.03507},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2012.03507},
}
Figure