Abstract
Although our pupils slightly dilate when we look at an intended target, they do not when we look at irrelevant distractors. This finding suggests that it may be possible to decode the intention of an observer, understood as the outcome of implicit covert binary decisions, from the pupillary dynamics over time. However, few previous works have investigated the feasibility of this approach and the few that did, did not control for possible confounds such as motor-execution, changes in brightness, or target and distractor probability. We report on our efforts to decode intentions from pupil dilation obtained under strict experimental control on a single trial basis using a machine learning approach. The basis for our analyses are data of 69 participants who looked at letters that needed to be selected with stimulus probabilities that varied systematically in a blockwise manner (n = 19,417 trials). We confirm earlier findings that pupil dilation is indicative of intentions and show that these can be decoded with a classification performance of up to 76% area under the curve for receiver operating characteristic curves if targets are rarer than distractors. To better understand which characteristics of the pupillary signal are most informative, we finally compare relative feature importances. The first derivative of pupil size changes was found to be most relevant, allowing us to decode intention within only about 800 ms of trial onset. Taken together, our results provide credible insights into the potential of decoding intentions from pupil dilation and may soon form the basis for new applications in visual search, gaze-based interaction, or human-robot interaction.
Introduction
Psychological research has long established that human pupils do more than regulate light; they dilate in response to arousal, cognitive load, and the identification of significant "targets." This physiological response suggests the potential for pupillometry as a hands-free communication channel. However, decoding these "intentions" from pupil data is challenging due to significant confounds.
In typical experimental settings, identifying a target is often coupled with a motor action, such as a button press, which itself triggers a large pupil dilation that can mask the subtler signal of the decision. Additionally, environmental factors like ambient light and the statistical probability of a target appearing (the "oddball effect") are rarely isolated. This study aims to extract the pure intentional signal from the noise of motor execution and environmental variables to determine if a machine learning model can reliably predict binary decisions on a single-trial basis.
Methods
The researchers employed a Random Forest (RF) classification approach, chosen for its ability to model non-linear relationships and provide interpretable feature importance rankings.
To isolate the decision signal from motor-related noise, the experiment included trials where participants responded with a key press and trials where they did not. Data was collected from 69 participants across 19,417 trials using an SMI Hi-Speed 1250 Eye tracker at 50 Hz. The experimental design systematically varied the probability of a target appearing (25%, 50%, or 75%) to assess the impact of expectancy on the pupillary signal.
The model utilized 26 features extracted from the pupillary signal, including:
- Simple metrics: Average size, variance, and absolute baseline diameter.
- Temporal features: Sampled z-standardized pupil sizes over a 1-second window.
- Kinetic features: The first derivative (velocity) of the pupil size change.
Evaluation was performed using nested cross-validation to ensure generalizability across different participants.
Results and Key Insights
The study yielded several critical findings regarding the nature of pupillary responses to intentional targets:
- The "Oddball" Advantage: Classification performance was highest when targets were rare (25% probability), achieving an Area Under the Curve (AUC) of up to 0.76. The "surprise" of an unexpected target appears to amplify the dilation signal.
- Velocity as a Primary Feature: Feature importance analysis revealed that the first derivative of pupil size—the velocity of the change—was more informative than absolute size. Specifically, changes occurring between 400ms and 800ms post-stimulus were the most predictive.
- Rapid Decoding: The model successfully decoded intentions within approximately 800 ms of stimulus onset, significantly faster than the time required for the pupil to reach its peak dilation.
- Motor Independence: While motor actions do affect pupil size, the "decision" signal remains detectable independently. Interestingly, the model showed slightly higher accuracy in conditions without a key press, suggesting that motor signals can act as noise in intent-decoding tasks.
Discussion and Future Work
While an AUC of 0.76 provides a strong proof-of-concept for decoding intentions, it is not yet sufficient for high-stakes applications like assistive wheelchair control. Current limitations include the need for strict lighting control and high-precision, expensive eye-tracking hardware.
Future research will focus on integrating pupil data with other gaze-based features, such as fixation duration, to improve accuracy. Such advancements could pave the way for practical human-computer interaction (HCI) systems and assistive technologies for individuals with severe motor impairments.
Summary updated