Using CognitIDE to Capture Developers' Cognitive Load via Physiological Activity During Everyday Software Development Tasks

Fabian Stolp, Charlotte Brandebusemeyer, Franziska Hradilak, Lara Kursawe, Magnus Menger, Franz Sauerwald, Bert Arnrich

Abstract

Measuring developer experience objectively remains a significant challenge in software engineering. Traditional methods rely on subjective surveys or static code metrics, which often fail to capture real-time mental effort. This paper introduces CognitIDE, an open-source IntelliJ plugin designed to capture physiological data—including heart rate, brain activity, and eye tracking—in real-time. By synchronizing these data streams with IDE interactions and mapping them directly onto source code, CognitIDE enables researchers to identify specific "friction points" in software development. A feasibility study demonstrates the tool's ability to objectively measure cognitive load during natural programming tasks.

Introduction

The mental effort required to understand and manipulate code, known as cognitive load, is a critical factor in developer productivity and well-being. However, existing measurement techniques are often intrusive or biased. Surveys interrupt the "flow" state, and static metrics like cyclomatic complexity do not account for individual differences in comprehension.

CognitIDE addresses this gap by providing a system that unobtrusively measures cognitive load within a natural programming environment. By integrating body sensor data with IDE events, the tool allows for a granular analysis of which functions or logic structures cause increased stress or mental fatigue.

Methods and System Architecture

The CognitIDE plugin is built for the IntelliJ platform and features a modular architecture designed for research flexibility. It utilizes the Lab Streaming Layer (LSL) as middleware to synchronize diverse data streams from various hardware sensors, including EEG, heart rate, and electrodermal activity (EDA).

Overview of the CognitIDE system integrating eye tracking and physiological sensors.

Automated Experimental Workflow

To minimize the "observer effect" and ensure study validity, CognitIDE automates the entire experimental protocol. The plugin handles consent forms, baseline measurements (via relaxation videos), task presentation, and post-task questionnaires like the NASA-TLX. This automation allows participants to work for extended periods without human moderator interruption.

The automated experimental workflow managed by the CognitIDE plugin.

Gaze-to-Code Mapping

A core feature of CognitIDE is its ability to map eye-tracking data directly to code symbols (e.g., classes, methods, variables). By calculating which symbols the developer is viewing and overlaying physiological "heat" onto those specific elements, researchers can pinpoint the exact lines of code that trigger high cognitive load.

Feasibility Study

The authors evaluated CognitIDE using real-world Java code from the Corona-Warn-App project (1,402 lines of code across 19 files). Four participants (students and research assistants, ages 23–29) performed four common tasks: coding, debugging, documentation, and email writing.

Key Results and Insights

  • Technical Performance: The plugin proved lightweight, generating less than 300 MB of data per hour while maintaining smooth IDE performance.
  • Physiological Validation: Data confirmed significantly higher cognitive load during programming tasks compared to baseline. Key indicators included increased pupil dilation, higher skin conductance responses, and characteristic shifts in EEG alpha (7-13 Hz) and theta (4-7 Hz) bands.
  • Symbol-Level Granularity: The most significant insight was the ability to link physiological stress to specific code structures, such as complex nested conditionals.
  • User Acceptance: Participants found the automated workflow non-disruptive and the gaze-to-code mapping accurate.

Limitations and Future Work

While the feasibility study demonstrated the tool's potential, several limitations remain. The small, homogeneous sample size necessitates further testing with professional developers. Additionally, while the IDE environment is natural, current EEG headsets are somewhat obtrusive; future research will explore more discreet wearables. Finally, the authors propose using CognitIDE to investigate the cognitive impact of AI-generated code suggestions, determining whether they truly reduce mental effort or introduce new forms of cognitive load during the review process.

Summary updated

BibTeX
@inproceedings{stolp2025using,
  author = {Fabian Stolp and Charlotte Brandebusemeyer and Franziska Hradilak and Lara Kursawe and Magnus Menger and Franz Sauerwald and Bert Arnrich},
  title = {{Using CognitIDE to Capture Developers' Cognitive Load via Physiological Activity During Everyday Software Development Tasks}},
  booktitle = {IDE@ICSE},
  year = {2025},
  doi = {10.1109/ide66625.2025.00014},
}
Figure