TL;DR
This paper introduces a novel method for measuring cognitive workload in manufacturing settings by combining pupillometry and heart-rate variability (HRV) data. The authors developed new cognitive-workload indices that better capture dynamic changes and align with task complexity and performance, offering a more robust approach than traditional methods.
What Problem Does This Paper Address?
The increasing digitalization of manufacturing, driven by Industry 4.0 and 5.0, places new demands on human operators. While physical workload has been well-studied, understanding and monitoring cognitive workload is crucial for maintaining operator well-being, preventing burnout, and optimizing performance in these complex, human-centric environments. Existing methods for assessing cognitive workload, such as self-reports or performance metrics, can be subjective or only capture workload over extended periods. Physiological measures offer a more objective approach, but previous studies often used single biomarkers or averaged data, which may not accurately reflect the dynamic nature of cognitive effort during complex tasks. This paper aims to bridge this gap by developing and validating a new method using multiple physiological signals to provide a more nuanced and accurate assessment of cognitive workload.
Methods
The core of this research lies in the development of novel cognitive-workload indices derived from physiological measurements. The authors utilized two primary sensors:
- Eye-tracking (Pupillometry): Specifically, they focused on pupil dilation metrics. Instead of using simple averaged pupil size, they employed an "epoch analysis" approach. This involved dividing the data into non-overlapping 7-second segments and calculating the average peak pupil dilation within each epoch. Peak pupil dilations above zero were converted into a "high cognitive-workload index" (1), and those below zero into a "low cognitive-workload index" (0). This method aims to capture more dynamic changes in pupil dilation.
- Electrocardiography (ECG) (Heart-Rate Variability - HRV): They extracted cardiac parameters, with a particular focus on PNN50 (the proportion of successive NN intervals differing by more than 50 ms). Similar to pupillometry, they used a non-overlapping 10-second epoch analysis. Mean baselined PNN50 values less than zero were converted to a "high cognitive-workload index" (1), and those greater than zero to a "low cognitive-workload index" (0), reflecting the inverse relationship between lower HRV and higher cognitive workload.
These individual indices were then accumulated for each task complexity level to create overall cognitive-workload indices. The study also investigated the influence of operator expertise as a mediating factor.
Datasets
The study used a custom dataset collected from 17 participants (6 experts, 11 non-experts) performing a Wankel Engine assembly task. This task was designed with three distinct complexity levels: low (housing assembly), medium (gear assembly), and high (rotor and cover assembly), in addition to a baseline rest condition. The data collected included:
- Pupil diameter and derivative from Tobii Pro Glasses 3 eye-tracking.
- ECG data for heart-rate variability from a Zephyr Bioharness 3 sensor.
- Assembly completion time as a measure of task performance.
This dataset represents a novel benchmark for evaluating cognitive workload in a simulated manufacturing assembly environment, specifically designed to vary task complexity and consider expertise.
Key Insights and Ideas
- Dynamic Indices are Key: Averaging physiological signals over long periods can mask important fluctuations. The epoch-based conversion into cognitive-workload indices provides a more sensitive measure of dynamic cognitive effort.
- Pupillometry and HRV Complement Each Other: While both sensors showed significant differences related to task complexity, their trends sometimes diverged, highlighting the need for a multi-modal approach. The developed indices aimed to align these trends.
- Expertise Matters, But Its Impact is Nuanced: While experts generally showed lower cognitive workloads, the statistical significance of expertise and its interaction with complexity was not always clear, suggesting that expertise might compensate for complexity in ways not fully captured by these specific metrics.
- Cognitive Workload Directly Impacts Performance: The study found a strong correlation between the developed cognitive-workload indices and task completion time, reinforcing the link between mental effort and operational efficiency.
Key Results
- Significant Complexity Effects: Both pupillometry and cardiac cognitive-load indices showed statistically significant differences across the four stages (rest, low, medium, high complexity).
- Pupillometry Trends: Baselined pupil dilation increased with task complexity, particularly in non-experts. Experts showed a marked increase from rest to medium complexity, with a slight decrease at high complexity, possibly indicating compensatory mechanisms.
- HRV Trends: Baselined PNN50 generally decreased with increasing complexity, indicating higher cognitive workload, though the trends between experts and non-experts showed some differences.
- Correlation with Performance: The developed cognitive-workload indices showed a strong correlation with assembly completion time, with higher workload generally leading to longer task durations.
- Expertise Differences: Non-experts consistently exhibited higher cognitive-load indices than experts across most complexity levels, suggesting a greater mental burden.
Limitations and Future Work
- Lack of Perceived Workload: The study did not include subjective self-reports of workload at short intervals, which could provide a richer understanding when compared with objective physiological data. Future research should incorporate this.
- Statistical Power: The study acknowledges a relatively low statistical power (0.3), which could limit the detection of subtle effects. Increasing the sample size in future studies is recommended.
- Distinguishing Low/Medium Complexity: The proposed indices, while effective for high complexity, struggled to clearly differentiate between low and medium complexity, especially for non-experts. This might be due to the bio-sensors' sensitivity or the specific nature of the assembly tasks.
- Future Directions: The authors suggest exploring the impact of additional personal traits on cognitive workload and designing more specialized scenarios to better verify the influence of expertise. Further research could also focus on real-time implementation of these indices for adaptive assistance systems in manufacturing.
Summary updated