Beyond performance: the role of task demand, effort, and individual differences in ab initio pilots

Mohammad-Javad Darvishi-Bayazi, Andrew Law, Sergio Mejía Romero, Sion Jennings, Irina Rish, Jocelyn FaubertScientific 2023 · DOI

TL;DR

This study moves beyond simple performance metrics in flight training to examine the "hidden" physiological cost of flying. By monitoring brain activity (EEG), heart rate, and pupil size, researchers found that while group trends are predictable, individual pilots vary wildly in how much effort they exert to achieve the same results—a distinction that could revolutionize personalized pilot training.

What Problem Does This Paper Address?

In aviation, human error accounts for nearly 70% of accidents. To mitigate this, flight training is rigorous, but it often relies on "outcome-based" metrics: did the student maintain altitude? Did they hit the target heading?

The problem is that performance alone doesn't tell the whole story. Two student pilots might achieve the same score on a flight maneuver, but one might be operating at their absolute cognitive limit (low efficiency), while the other is comfortably in control (high efficiency). If a task becomes slightly harder, the first pilot may "redline" and fail, while the second has plenty of spare capacity. This paper seeks to quantify this "mental effort" using physiological sensors to help instructors understand a student's true expertise and readiness for more complex tasks.

Methods

The researchers recruited 24 ab initio (novice) pilots and placed them in a high-fidelity Marinvent flight simulator. Participants performed maneuvers across three difficulty levels:

  • D1 (Simple): Single-direction change in one axis (e.g., just climbing).
  • D2 (Moderate): A change with a reversal in one axis (e.g., climbing then leveling off).
  • D3 (Complex): Simultaneous changes in two axes (e.g., turning while climbing/descending).

To capture the "internal" state of the pilots, the team used a multi-modal approach:

  • EEG (Electroencephalography): Specifically looking at frontal theta power (F2 and F4 channels), a known marker for mental workload.
  • ECG (Electrocardiography): To track mean Heart Rate (HR).
  • Pupillometry: Measuring pupil diameter changes via an eye tracker.
  • NASA-TLX: A standard subjective questionnaire where pilots rate their own perceived effort and frustration.
  • Control Activity: Measuring how much the pilot moved the "yoke" (stick) to maintain control.

Datasets

The study utilized a primary dataset collected by the authors specifically for this research. It involved 24 volunteers (13 women, 11 men) aged 21–41. While no external benchmark datasets were used, the study provides a framework for how flight schools might collect and utilize "biometric" datasets to track student progress over time.

Key Insights and Ideas

  • The "Efficiency" Gap: The most striking insight is that group-level averages are misleading. While the group showed a correlation between difficulty and heart rate, individual plots showed that some pilots' performance crashed even when their physiological effort remained high, while others maintained performance by "ramping up" their mental effort.
  • Frontal Theta as a "Mental Tachometer": Frontal theta brain waves proved to be a sensitive indicator of task demand. Interestingly, for some participants, theta power actually decreased during the hardest tasks, suggesting they may have "disengaged" or given up when the demand exceeded their capacity.
  • Subjective vs. Objective: Pilots were surprisingly bad at rating their own performance. Nearly half of the participants misinterpreted the NASA-TLX performance scale, highlighting why objective physiological monitoring is more reliable than self-reporting.
  • Control Activity as a Proxy: There was a high correlation (R² = 0.54) between how much a pilot moved the controls and their perceived mental demand. This suggests that even without expensive EEG, "fidgety" control movements can be a strong indicator of high workload.

Key Results

  • Difficulty Scaling: Dual-axis maneuvers (D3) caused a significant spike in all metrics compared to D1 and D2.
  • Heart Rate Consistency: Heart rate increased from D2 to D3 for all 24 participants, making it the most consistent (though perhaps least specific) marker of increased demand.
  • Sub-grouping for Training: When the researchers split the pilots into sub-groups based on their physiological responses, they found that pilots who showed the highest increase in EEG frontal theta (more effort) actually had the smallest decrease in performance. This identifies them as "high-effort/high-performance" students who are successfully meeting the challenge.
  • Performance Error (RMSE): Performance error significantly increased with difficulty ($X^2(2) = 29.25, p < 0.001$), but the physiological data explained why some pilots failed while others succeeded.

Limitations and Future Work

The authors acknowledge several hurdles for real-world application:

  • Data Noise: Pupillometry data was frequently lost due to head movements or blinks, making it difficult to use in a dynamic cockpit environment.
  • Single Session: The study only looked at a single point in time. Future work needs to track these physiological markers over weeks of training to see if "neural efficiency" improves as a student becomes an expert.
  • Causality: It remains to be seen if providing this physiological feedback to instructors in real-time actually leads to better-trained pilots.

In short, the paper argues that the future of flight training isn't just about watching the plane—it's about watching the pilot's brain.

Summary updated

Abstract

Aviation safety depends on the skill and expertise of pilots to meet the task demands of flying an aircraft in an effective and efficient manner. During flight training, students may respond differently to imposed task demands based on individual differences in capacity, physiological arousal, and effort. To ensure that pilots achieve a common desired level of expertise, training programs should account for individual differences to optimize pilot performance. This study investigates the relationship between task performance and physiological correlates of effort in ab initio pilots. Twenty-four participants conducted a flight simulator task with three difficulty levels and were asked to rate their perceived demand and effort using the NASA TLX. We recorded heart rate, EEG brain activity, and pupil size to assess changes in the participants' mental and physiological states across different task demands. We found that, despite group-level correlations between performance error and physiological responses, individual differences in physiological responses to task demands reflected different levels of participant effort and task efficiency. These findings suggest that physiological monitoring of student pilots might provide beneficial insights to flight instructors to optimize pilot training at the individual level.
BibTeX
@inproceedings{darvishibayazi2023beyond,
  author = {Mohammad-Javad Darvishi-Bayazi and Andrew Law and Sergio Mejía Romero and Sion Jennings and Irina Rish and Jocelyn Faubert},
  title = {{Beyond performance: the role of task demand, effort, and individual differences in ab initio pilots}},
  booktitle = {Scientific},
  year = {2023},
  doi = {10.1038/s41598-023-41427-4},
}

Loading related content...

Figure