An error-aware gaze-based keyboard by means of a hybrid BCI system

Fotis P. Kalaganis, Elisavet Chatzilari, Spiros Nikolopoulos, Ioannis Kompatsiaris, Nikolaos LaskarisScientific 2018 · DOI

Abstract

Gaze-based keyboards offer a flexible human-computer interaction (HCI) method for both disabled and able-bodied individuals. However, these systems are often error-prone due to eye-tracking misinterpretations and calibration drifts, particularly in "fast mode." This paper presents a novel bimodal error detection system (EDS) that aggregates decisions from two distinct data streams: gaze-related measurements and electroencephalography (EEG). By detecting Error-Related Potentials (ErrPs) and specific eye-transition patterns, the system can automatically flag and correct typos. Experimental results from 10 participants demonstrate that this hybrid approach significantly reduces typesetting time and improves overall system utility.

Introduction

Gaze-based keyboards are essential communication tools for individuals with severe motor disabilities, such as those with ALS. Despite their utility, they suffer from the "Midas touch" problem—where accidental selections occur due to lingering gaze—and calibration inaccuracies. In high-speed typing scenarios, the traditional "detect-react-correct" cycle, where a user must manually navigate to a backspace key after realizing a mistake, becomes a major bottleneck. This research aims to automate error correction by identifying the user's internal realization of an error the moment it happens.

The Bimodal Error Detection System

The proposed system utilizes a hybrid Brain-Computer Interface (hBCI) architecture to process two independent signals simultaneously.

Overview of the hybrid BCI system architecture, integrating EEG and gaze data streams for error detection.

1. Neural Signal Processing (EEG)

The system monitors for Error-Related Potentials (ErrPs), which are neurophysiological signatures appearing 300–400ms after a subject perceives an error. To extract these signals from the noisy EEG data (often contaminated by eye-movement artifacts), the researchers employed Discriminant Spatial Patterns (DSP) for spatial filtering. This ensures that the ErrP is isolated as a distinct event rather than a byproduct of gaze shifts.

2. Gaze Trajectory Analysis

Beyond simple coordinates, the system analyzes the dynamics of eye movements using Hjorth Descriptors (Activity, Mobility, and Complexity). These parameters describe the speed and acceleration of the gaze. When a selection is correct, the eyes move predictably toward the next target; when an error occurs, the gaze often lingers or exhibits "micro-adjustments" as the user reacts to the mistake.

3. Late Fusion Classification

Separate Support Vector Machine (SVM) classifiers were trained for the EEG and gaze streams. The system uses a "late fusion" approach, combining the confidence scores of both models to make a final determination. This method leverages the high specificity of EEG (reducing false deletions) and the high sensitivity of gaze patterns (catching more typos).

Experimental Methodology

The study involved 10 participants who typed 20 pre-memorized sentences. The hardware setup included a 64-channel EEG and a 30Hz eye-tracker.

The virtual gaze-based keyboard interface used in the study, featuring a high-contrast layout for eye-tracking.

To address the class imbalance—since typos occur less frequently than correct entries—the researchers utilized the Synthetic Minority Oversampling Technique (SMOTE) to generate synthetic error samples for training. A fixed dwell time of 0.5 seconds was used to simulate a "fast mode" typing environment.

Key Results and Discussion

The hybrid system demonstrated a clear advantage over traditional gaze-only keyboards:

  • Efficiency Gains: The system achieved a 9.3% increase in typing speed, saving an average of 2.7 seconds per sentence.
  • Classification Accuracy: The late fusion model reached a sensitivity of approximately 86% and a specificity of 83%.
  • Utility Metric: The authors prioritized a "Utility metric" that penalizes false positives (deleting a correct letter) more heavily than false negatives, as the former is more disruptive to the user experience.

Crucially, the study confirmed that ErrPs can be robustly detected even during active eye-tracking, debunking concerns that eye artifacts would render the neural signal unusable. Users who were naturally more error-prone saw the greatest benefit, suggesting the system is highly effective for those who struggle with eye-tracker calibration.

Conclusion and Future Work

While the results are promising, the study's small sample size of able-bodied participants necessitates further validation with the target demographic of individuals with severe motor impairments. Future research should explore the integration of linguistic auto-correct features and the transition from research-grade 64-channel EEG to consumer-grade wearables to make this technology accessible for daily use.

Summary updated

Abstract

Gaze-based keyboards offer a flexible way for human-computer interaction in both disabled and able-bodied people. Besides their convenience, they still lead to error-prone human-computer interaction. Eye tracking devices may misinterpret user's gaze resulting in typesetting errors, especially when operated in fast mode. As a potential remedy, we present a novel error detection system that aggregates the decision from two distinct subsystems, each one dealing with disparate data streams. The first subsystem operates on gaze-related measurements and exploits the eye-transition pattern to flag a typo. The second, is a brain-computer interface that utilizes a neural response, known as Error-Related Potentials (ErrPs), which is inherently generated whenever the subject observes an erroneous action. Based on the experimental data gathered from 10 participants under a spontaneous typesetting scenario, we first demonstrate that ErrP-based Brain Computer Interfaces can be indeed useful in the context of gaze-based typesetting, despite the putative contamination of EEG activity from the eye-movement artefact. Then, we show that the performance of this subsystem can be further improved by considering also the error detection from the gaze-related subsystem. Finally, the proposed bimodal error detection system is shown to significantly reduce the typesetting time in a gaze-based keyboard.
BibTeX
@inproceedings{kalaganis2018error,
  author = {Fotis P. Kalaganis and Elisavet Chatzilari and Spiros Nikolopoulos and Ioannis Kompatsiaris and Nikolaos Laskaris},
  title = {{An error-aware gaze-based keyboard by means of a hybrid BCI system}},
  booktitle = {Scientific},
  year = {2018},
  doi = {10.1038/s41598-018-31425-2},
}
Figure