SSVEP BCI and Eye Tracking Use by Individuals With Late-Stage ALS and Visual Impairments

Betts Peters, Steven Bedrick, Shiran Dudy, Brandon Eddy, Matt Higger, Michelle Kinsella, Deirdre McLaughlin, Tab Memmott, Barry Oken, Fernando Quivira, Scott Spaulding, Deniz Erdoğmuş, Melanie Fried‐OkenFrontiers in Human Neuroscience 2020 · DOI

TL;DR

This study investigates the effectiveness of a modified eye-tracking and SSVEP BCI system, integrated with the Shuffle Speller interface, for individuals with late-stage ALS and visual impairments. The findings suggest that this approach holds promise for improving communication access for this challenging population, outperforming traditional eye-tracking methods.

What Problem Does This Paper Address?

Individuals with late-stage Amyotrophic Lateral Sclerosis (ALS) often experience a progressive loss of voluntary motor control, leading to severe communication difficulties. While eye-tracking systems are commonly used for augmentative and alternative communication (AAC), their effectiveness can be significantly hampered by concomitant visual or ocular motility impairments, which are common in advanced ALS. These impairments can include reduced visual acuity, dry eyes, ptosis (drooping eyelids), and difficulties with eye movements. Furthermore, some individuals may progress to a "locked-in" state where even eye movements become impossible. This paper addresses the critical need for robust and adaptable communication solutions for this population by exploring the potential of combining a novel typing interface (Shuffle Speller) with both modified eye-tracking and Steady-State Visual Evoked Potential (SSVEP) Brain-Computer Interfaces (BCIs).

Methods

The study employed a single-case experimental research design, specifically an alternating treatments design, to compare three typing conditions for two participants with late-stage ALS and visual impairments:

  1. Baseline (Commercial Eye Tracking): A standard, commercially available eye-tracking communication software (Communicator 5) with a 1.2-second dwell time for selection.
  2. Modified Eye Tracking (Shuffle Speller + Eye Tracking - SSET): The Shuffle Speller interface, which adapts stimulus presentation based on user input, combined with eye-tracking technology. This condition used a modified eye-tracking calibration and a 2-second dwell time.
  3. SSVEP BCI (Shuffle Speller + SSVEP BCI - SSBCI): The Shuffle Speller interface controlled via SSVEP BCI. SSVEP responses were elicited by flashing LED arrays, and the system used a recursive Bayesian inference model for character selection.

The Shuffle Speller interface itself is designed to adapt to user input characteristics by adjusting stimulus presentation. For this study, it was configured with larger font sizes, high contrast, and contextual cues to accommodate visual impairments. The SSVEP BCI utilized EEG data to detect visual evoked potentials, with adaptive trial lengths determined by calibration data.

Datasets

The study did not use pre-existing datasets for training or evaluation. Instead, it involved data collection from two participants with late-stage ALS. The "dataset" generated for this study consists of copy-spelling performance data collected over multiple sessions for each participant across the different experimental conditions. The words used for copy-spelling were drawn from the SUBTLEXus dataset, a corpus of English words.

Key Insights and Ideas

  • Adaptability is Key: The Shuffle Speller interface's ability to adapt stimulus presentation based on user input appears crucial for individuals with variable performance due to fatigue or fluctuating attention.
  • SSVEP BCI Shows High Potential: Despite variability, SSVEP BCI demonstrated the highest maximum accuracies for both participants, suggesting it can be a viable, albeit potentially slower, communication channel.
  • Visual Impairments are a Major Hurdle: The study strongly reinforces that visual impairments significantly impact the effectiveness of standard eye-tracking systems, necessitating specialized approaches.
  • Individualized Approaches are Essential: The variability in performance, even within the same condition, highlights the need for highly personalized BCI and AAC systems.
  • Hybrid Systems Might Be the Future: The authors suggest that hybrid systems, combining the most reliable control signals (e.g., eye-tracking and SSVEP), could offer greater resilience and improved performance.

Key Results

  • Participant 1: Was unable to type any correct characters with the commercial eye-tracking system. Achieved accuracies of up to 50% with Shuffle Speller eye tracking (SSET) and up to 89% with Shuffle Speller SSVEP BCI (SSBCI). Mean accuracy for SSET was 31.9%, and for SSBCI was 57.0%.
  • Participant 2: Achieved higher maximum accuracies with Shuffle Speller, typing with up to 63% accuracy with SSET and 100% accuracy with SSBCI. Average baseline accuracy was 29.0%, with C2.5 at 39.0%. Mean SSBCI accuracy was 65.0%.
  • Variability: Typing accuracy for both Shuffle Speller conditions (SSET and SSBCI) was highly variable for both participants, particularly in the SSBCI condition.
  • User Preference: Both participants preferred the SSBCI system due to higher accuracy and a feeling of control, despite some discomfort from flashing LEDs. Participant 2 rated SSBCI highest for workload and comfort.

Limitations and Future Work

  • Small Sample Size: The study involved only two participants, making it difficult to draw broad conclusions and highlighting the need for larger studies.
  • High Variability: The significant variability in performance, especially in the SSBCI condition, makes direct comparisons challenging and suggests that further optimization is needed.
  • Data Collection Challenges: Factors like participant fatigue, illness, and potential equipment issues (e.g., SSBCI code error, EEG signal quality) impacted data collection and analysis.
  • Typing Speed: While accuracy was a focus, typing speed was generally slower with SSVEP BCI compared to eye-tracking, which is a common trade-off.
  • Future Work: The authors suggest future research should include more participants with late-stage ALS and other SSPI conditions, explore hybrid BCI-ET input signals, and develop custom interface designs to better suit individual visual abilities and limitations. They also propose investigating alternative SSVEP stimulation methods to increase comfort and exploring real-time signal display for operators to quickly address poor signal quality. The use of "number of target characters copied" as a primary dependent variable in future studies is also recommended.

Summary updated

Abstract

Access to communication is critical for individuals with late-stage amyotrophic lateral sclerosis (ALS) and minimal volitional movement, but they sometimes present with concomitant visual or ocular motility impairments that affect their performance with eye tracking or visual brain-computer interface (BCI) systems. In this study, we explored the use of modified eye tracking and steady state visual evoked potential (SSVEP) BCI, in combination with the Shuffle Speller typing interface, for this population. Two participants with late-stage ALS, visual impairments, and minimal volitional movement completed a single-case experimental research design comparing copy-spelling performance with three different typing systems: (1) commercially available eye tracking communication software, (2) Shuffle Speller with modified eye tracking, and (3) Shuffle Speller with SSVEP BCI. Participant 1 was unable to type any correct characters with the commercial system, but achieved accuracies of up to 50% with Shuffle Speller eye tracking and 89% with Shuffle Speller BCI. Participant 2 also had higher maximum accuracies with Shuffle Speller, typing with up to 63% accuracy with eye tracking and 100% accuracy with BCI. However, participants' typing accuracy for both Shuffle Speller conditions was highly variable, particularly in the BCI condition. Both the Shuffle Speller interface and SSVEP BCI input show promise for improving typing performance for people with late-stage ALS. Further development of innovative BCI systems for this population is needed.
BibTeX
@inproceedings{peters2020ssvep,
  author = {Betts Peters and Steven Bedrick and Shiran Dudy and Brandon Eddy and Matt Higger and Michelle Kinsella and Deirdre McLaughlin and Tab Memmott and Barry Oken and Fernando Quivira and Scott Spaulding and Deniz Erdoğmuş and Melanie Fried‐Oken},
  title = {{SSVEP BCI and Eye Tracking Use by Individuals With Late-Stage ALS and Visual Impairments}},
  booktitle = {Frontiers in Human Neuroscience},
  year = {2020},
  doi = {10.3389/fnhum.2020.595890},
}
Figure