Less Stress, More Privacy: Stress Detection on Anonymized Speech of Air Traffic Controllers

Janaki Viswanathan, Alexander Blatt, Konrad Hagemann, Dietrich KlakowarXiv 2025 · DOI · arXiv

Abstract

Monitoring stress in Air Traffic Controllers (ATCOs) is critical for aviation safety, yet privacy regulations like GDPR pose significant hurdles for voice-based analysis. This study demonstrates that deep learning architectures, specifically Convolutional Recurrent Neural Networks (CRNNs) with Attention, can accurately detect stress even in anonymized speech. Achieving up to 93.6% accuracy on benchmark datasets and 80.1% on real-world simulations, the research proves that privacy-preserving measures do not necessarily compromise safety monitoring capabilities.

Introduction

Air Traffic Control is a high-stakes environment where excessive stress can lead to catastrophic errors. While physiological sensors offer one way to monitor stress, they are often too intrusive for daily operational use. Speech analysis provides a non-intrusive alternative; however, voice data is classified as personal information under GDPR. To comply with privacy laws, this data must often be anonymized. The central question is whether stripping a voice of its identity also removes the acoustic cues necessary for stress detection.

Methodology

The researchers evaluated three deep learning architectures to identify the most effective approach for stress classification:

  1. CNN: Used to extract spectral features.
  2. CRNN: Combines CNNs with Long Short-Term Memory (LSTM) networks to capture temporal patterns.
  3. CRNN + Attention: Incorporates a multi-head attention layer to focus the model on the most salient "stress-heavy" segments of an utterance.
Proposed CRNN + Attention architecture for stress detection from speech features.

Input features compared included Log Mel Spectrograms (LMS) and Mel Frequency Cepstral Coefficients (MFCC). For anonymization, the team employed Lightweight Voice Anonymization (LVA), which uses waveform resampling to alter vocal tract characteristics while preserving pitch, effectively masking the speaker's identity.

Datasets

The models were tested on two distinct datasets:

  • SUSAS: A benchmark containing over 5,000 samples across nine speaking styles.
  • DFS-MAS: A realistic dataset from the German Air Navigation Service featuring ATCOs in 90-minute simulations. Due to its high class imbalance (60 stress vs. 678 no-stress samples), heavy data augmentation using white noise and vocal tract perturbation was required.

Results and Discussion

The study revealed several critical insights regarding the intersection of privacy and performance.

Performance and Anonymization

Surprisingly, anonymization acted as a form of data augmentation on the DFS-MAS dataset, improving generalization by preventing the model from overfitting to specific voices. On the SUSAS dataset, the "accuracy tax" for anonymization was minimal, with only a 1-2% drop compared to original data.

Performance comparison across different models, features, and anonymization states on SUSAS and DFS-MAS datasets.

Feature Robustness

While LMS generally performed better for intra-dataset tasks, MFCCs proved superior for cross-domain scenarios (e.g., training on lab data and testing on simulations). MFCCs provide robust "information condensation," which helps bridge the gap between different recording environments.

The "Neutral" Trap

The primary challenge post-anonymization was the misclassification of "neutral" speech. The anonymization process occasionally caused the model to mistake calm speech for "clear" or "slow" styles due to shifts in formants.

Conclusion

This research confirms that high-accuracy stress detection is feasible on anonymized ATCO speech. The CRNN + Attention model reached a peak accuracy of 93.6% on anonymized benchmark data and 80.1% on real-world simulations. Future work will focus on transfer learning from larger speech datasets and optimizing these models for real-time deployment on edge devices within control towers.

Summary updated

Abstract

Air traffic control (ATC) demands multi-tasking under time pressure with high consequences of an error. This can induce stress. Detecting stress is a key point in maintaining the high safety standards of ATC. However, processing ATC voice data entails privacy restrictions, e.g. the General Data Protection Regulation (GDPR) law. Anonymizing the ATC voice data is one way to comply with these restrictions. In this paper, different architectures for stress detection for anonymized ATCO speech are evaluated. Our best networks reach a stress detection accuracy of 93.6% on an anonymized version of the Speech Under Simulated and Actual Stress (SUSAS) dataset and an accuracy of 80.1% on our anonymized ATC simulation dataset. This shows that privacy does not have to be an impediment in building well-performing deep-learning-based models.
BibTeX
@misc{viswanathan2025less,
  author = {Janaki Viswanathan and Alexander Blatt and Konrad Hagemann and Dietrich Klakow},
  title = {{Less Stress, More Privacy: Stress Detection on Anonymized Speech of Air Traffic Controllers}},
  year = {2025},
  eprint = {2507.08882},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2507.08882},
}
Figure