Editorial: Detection and Estimation of Working Memory States and Cognitive Functions Based on Neurophysiological Measures

Felix Putze, Christian Mühl, Fabien Lotte, Stephen Fairclough, Christian HerffFrontiers in Human Neuroscience 2018 · DOI · OpenAlex

Introduction

Executive cognitive functions, such as working memory, are fundamental to human performance in complex tasks including problem-solving, navigation, and strategic planning. The ability to estimate these states—specifically working memory load or capacity—from neurophysiological and psychophysiological signals opens the door to adaptive systems. Such systems could dynamically respond to an operator's cognitive state, for instance, by simplifying a tutoring interface when a student is overloaded or filtering distractions during high-stakes operations. This editorial synthesizes a Research Topic in Frontiers in Human Neuroscience that aims to bridge the gap between laboratory-based neural correlates and real-time, single-trial estimation of cognitive functions in ecologically valid, "real-world" scenarios.

The Challenge of Ecological Validity

A primary hurdle in neuroergonomics is the transition from identifying neural "correlates" in controlled environments to building robust classifiers for complex, noisy settings. Traditional research often relies on repetitive tasks in sterile labs. However, the practical application of this research requires detecting cognitive states—like a pilot’s workload in a vibrating cockpit or a driver’s attention during a commute—using short segments of data. The goal is to move toward "closed-loop" systems where the interface senses the user's mental state and automatically adjusts to optimize performance and safety.

Multimodal Methodologies and Signal Processing

The research presented in this collection highlights a diverse array of sensing modalities and machine learning techniques:

  • Sensor Modalities: Electroencephalography (EEG) remains a cornerstone due to its high temporal resolution. However, researchers are increasingly integrating functional near-infrared spectroscopy (fNIRS) to capture prefrontal metabolic activity, alongside eye-tracking for visual attention and peripheral physiological measures like heart and breath rates.
  • Signal Processing & Machine Learning: Contributed studies utilize Support Vector Machines (SVM), graph-theoretic connectivity features to map functional brain networks, and Event-Related Potential (ERP) analysis.
  • Advanced Approaches: Significant focus is placed on multimodal fusion (e.g., EEG + fNIRS) to enhance classification robustness. Furthermore, transfer learning and cross-subject modeling are explored to reduce the extensive calibration time typically required for brain-computer interfaces (BCIs).

Key Insights: Beyond Single Modalities

Several critical themes emerge from the collective findings:

  1. The Advantage of Multimodality: Combining different signal types (e.g., electrical and metabolic) consistently improves estimation accuracy. These sensors provide complementary perspectives on the brain's response to cognitive demand.
  2. Contextual Interference: Real-world cognitive states are rarely "pure." Emotional states (valence) can significantly interfere with the neural markers of workload, complicating detection outside the lab.
  3. Real-World Feasibility: Studies such as those by Gateau et al. have successfully demonstrated workload classification in pilots during actual flight, proving that neurophysiological sensors can function effectively despite environmental noise and movement artifacts.
  4. Early Warning Systems: Research into "mind wandering" suggests that specific neural markers, such as increased alpha activity and attenuated auditory P3a components, can serve as early indicators of distraction in tasks like driving.

Results and Performance Metrics

The Research Topic demonstrates that binary and three-level workload classification is achievable with "better than chance" accuracy using single-trial, multimodal data. Key findings include:

  • Predictive Markers: Fronto-central spectral entropy was identified as a reliable predictor of performance in repeated memory tasks.
  • Calibration Efficiency: Cross-subject learning techniques allowed for high classification accuracy while minimizing the need for user-specific training data, a major step toward consumer-ready BCIs.

Limitations and Future Directions

Despite these advancements, several challenges remain. The "Lab-to-Life" gap persists, as movement artifacts and environmental noise continue to plague "in the wild" data collection. Additionally, there is a conceptual need to more clearly distinguish between "mental workload" and "memory load," as they may involve distinct underlying neural processes.

Perhaps most importantly, the field must move beyond mere detection to the design of effective adaptations. Even when a system accurately identifies a state of overload, the resulting intervention must be carefully crafted to ensure it actually improves the user's performance and experience. Future research should focus on the human-centered design of these closed-loop responses to ensure they provide meaningful support in high-pressure environments.

Summary updated

Abstract

Executive cognitive functions like working memory determine the success or failure of a wide variety of different cognitive tasks, such as problem solving, navigation, or planning. Estimation of constructs like working memory load or memory capacity from neurophysiological or psychophysiological signals would enable adaptive systems to respond to cognitive states experienced by an operator and trigger responses designed to support task performance (e.g., by simplifying the exercises of a tutor system when the subject is overloaded Gerjets et al., 2014, or by shutting down distractions from the mobile phone). The determination of cognitive states like working memory load is also useful for automated testing/assessment or for usability evaluation. While there exists a large body of research work on neural and physiological correlates of cognitive functions like working memory activity, fewer publications deal with the application of this research with respect to single-trial detection and real-time estimation of cognitive functions in complex, realistic scenarios. Single-trial classifiers based on brain activity measurements such as electroencephalography (EEG, Kothe and Makeig, 2011; Lotte et al., 2018), functional near-infrared spectroscopy (fNIRS, Putze et al., 2014; Herff et al., 2015), physiological signals (Fairclough et al., 2005; Fairclough, 2008), or eye tracking (Putze et al., 2013) have the potential to classify affective (Koelstra et al., 2010; Heger et al., 2014; Mühl et al., 2014) or cognitive states based upon short segments of data. For this purpose, signal processing and machine learning techniques need to be developed and transferred to real-world user interfaces. The goal of this Frontiers Research Topic was to advance the State-of-the-Art in signal-based modeling of cognitive processes. We were especially interested in research toward more complex and realistic study designs, for example collecting data in the wild or investigating the interaction between different cognitive processes or signal modalities. Bringing together many contributions in one format allowed us to look at the state of convergence or diversity regarding concepts, methods, and paradigms. The accepted manuscripts in this research topic cover a large range of aspects of cognition, reflecting the broadness of the field and its many application domains. A dominant challenge in the research topic is the analysis of cognitive workload (or memory load) from neurological signals. This does not come as a surprise because workload is a thoroughly studied construct and workload models can be immediately exploited, e.g., for adaptive human-machine interaction. While all these manuscripts share a joint research interest, they tackle the challenge of workload modeling in different application domains, with different signals, different classification approaches, and different features. Working memory and attentional control represent two recurring themes through the collection of papers in this research theme. The most prominent modality in this research topic is EEG, drawing from both spectral as well as time-domain features. In multiple articles, EEG is complemented by other modalities: Two of them use fNIRS as a different mode to capture neural activity (two others use fNIRS as single modality); two others use eye tracking as a way to capture visual attention and one also uses physiological signals (such as heart rate and breath rate). This shows that researchers today routinely select which (combinations of) signals are most promising for a given task.
BibTeX
@inproceedings{putze2018editorial,
  author = {Felix Putze and Christian Mühl and Fabien Lotte and Stephen Fairclough and Christian Herff},
  title = {{Editorial: Detection and Estimation of Working Memory States and Cognitive Functions Based on Neurophysiological Measures}},
  booktitle = {Frontiers in Human Neuroscience},
  year = {2018},
  doi = {10.3389/fnhum.2018.00440},
}
Figure