Abstract
Concealing the identity through speaker anonymization is essential in various situations. This study focuses on investigating how stuttering affects the anonymization process. Two scenarios are considered: preserving the pathology in the diagnostic/remote treatment context and obfuscating the pathology. The paper examines the effectiveness of three state-of-the-art approaches in achieving high anonymization, as well as the preservation of dysfluencies. The findings indicate that while a speaker conversion method may not achieve perfect anonymization (Baseline 27.25% EER and F0 Delta 32.63% EER), it does preserve the pathology. This effect was objectively evaluated by performing a stuttering classification. Although this solution may be useful in a remote treatment scenario for speech pathologies, it presents a vulnerability in anonymization. To address this issue, we propose an alternative approach that uses automatic speech recognition and text-based speech synthesis to avoid re-identification (48.27% EER).
Introduction
Speech is a rich biometric containing not only linguistic content but also paralinguistic information such as timbre, pitch, and unique temporal patterns. For individuals who stutter (People Who Stutter, PWS), these dysfluencies—including blocks, prolongations, and repetitions—can serve as a unique "vocal fingerprint." This vulnerability poses a significant challenge for eHealth applications, where a balance must be struck between protecting speaker privacy (e.g., for GDPR compliance) and preserving clinical utility for remote therapy and diagnosis.
This paper investigates the feasibility of anonymizing stuttered speech while either preserving or obfuscating the underlying pathology. We evaluate state-of-the-art voice conversion and re-synthesis techniques to determine their efficacy in removing speaker identity while maintaining the integrity of stuttering events.
Methods
We evaluated three primary approaches to speaker anonymization:
- StarGAN-v2VC (Baseline): A generative adversarial network (GAN) designed for many-to-many voice conversion. It aims to transform the source speaker's voice into a target style while preserving linguistic content.
- StarGAN with Modified Losses: To better capture the nuances of stuttered speech, we introduced pitch-modulation (F0) and intensity losses. These additions help the model retain the subtle intonations and volume changes characteristic of dysfluencies.
- Re-Synthesis (STT + TTS): A "hard" anonymization pipeline where speech is transcribed to text using OpenAI’s Whisper (STT) and then synthesized back into speech using FastPitch (TTS).
The performance was measured using ECAPA-TDNN embeddings for speaker verification (Equal Error Rate, EER) and a wav2vec 2.0-based classifier to evaluate the preservation of dysfluency types across three datasets: KSF-C, SEP-28k-Extended, and FluencyBank.
Results and Discussion
Our results highlight a clear trade-off between anonymity and utility.
Anonymization Strength
The original stuttered speech data exhibited an EER of 8.24%, indicating high re-identification risk. The re-synthesis (STT+TTS) approach provided the most robust protection, achieving an EER of 48.27%, which is close to the theoretical maximum of 50% (random guessing). In contrast, the StarGAN-based methods achieved EERs between 27.25% and 32.63%, suggesting that unique temporal patterns of stuttering may still allow for re-identification even after voice conversion.
Pathology Preservation
While re-synthesis excels at privacy, it often fails to preserve non-textual dysfluencies like blocks or prolongations. However, a surprising finding was that Whisper often transcribes repetitions literally (e.g., transcribing "cat" as "C-C-C-C-C-C cat"), allowing some stuttering artifacts to survive the text-based pipeline.
The StarGAN-v2VC model with F0 Delta losses proved superior for clinical utility, maintaining high recall for various dysfluency classes. For instance, "modifications" (Mod.) were preserved with high accuracy, as shown in the confusion matrix below.
Interestingly, training stuttering detection models on anonymized data appeared to act as a form of normalization, improving performance on original data by helping the model focus on dysfluency patterns rather than speaker-specific vocal characteristics.
Conclusion
This study demonstrates that speaker anonymization for stuttered speech is possible but requires careful selection of methods based on the use case. For remote therapy, voice conversion (StarGAN) preserves the pathology well but leaves a privacy gap. For maximum privacy, re-synthesis is preferred, though it may strip away critical clinical information. Future work will explore phoneme-based synthesis to bridge this gap, aiming to preserve the "how" of speech without compromising the "who."
Summary updated