SAIC: Integration of Speech Anonymization and Identity Classification

Ming Cheng, Xingjian Diao, Shitong Cheng, Wenjun LiuarXiv 2023 · DOI · arXiv

Abstract

Speech data contains unique biometric identifiers, posing significant privacy risks in digital health contexts. This paper introduces SAIC (Speech Anonymization and Identity Classification), a CNN-based pipeline that disentangles speaker identity from spoken content. Through a novel two-stage training process involving latent optimization and encoder training, SAIC achieves high-quality speech anonymization while simultaneously reaching a state-of-the-art 96.1% top-1 accuracy in speaker identity classification on the VoxCeleb1 dataset, outperforming several prominent Transformer-based architectures.

Introduction

In the era of telehealth and digital health records, voice data represents a significant privacy challenge. A person’s "voiceprint" serves as a unique biometric identifier. Protecting patient privacy requires systems capable of anonymizing speech—stripping away the speaker's identity—while preserving the content for medical documentation. Existing research often treats these as separate tasks and frequently relies on data-intensive Transformer models. SAIC addresses these challenges with an efficient, integrated CNN-based pipeline.

Methods

The SAIC architecture utilizes a two-stage training process to ensure effective feature disentanglement.

Two-Stage Training Process

  1. Stage 1: Latent Optimization: The system optimizes embeddings for content and identity in a latent space. A Fusion Decoder (FD) is trained to reconstruct audio from these embeddings. Gaussian noise is applied to content embeddings to ensure they do not retain identity information.
  2. Stage 2: Encoder Training: Using the optimized embeddings and the trained decoder from Stage 1, a Content Encoder (CE) and a Speaker Encoder (SE) are trained to map raw audio to the established embedding spaces using Mean Squared Error (MSE) loss.
The two-stage training process of SAIC, involving latent optimization and encoder training.

Architecture and Dataset

Instead of Transformers, the authors employ Residual Blocks (ResBlocks), arguing that CNNs are more efficient at capturing local temporal features in audio. The model was trained on VoxCeleb1, a large-scale dataset containing over 150,000 audio clips from 1,251 speakers, providing a diverse range of real-world acoustic environments.

Results

SAIC demonstrates superior performance in both identity classification and feature disentanglement.

Performance Comparison

The model achieved a 96.1% top-1 accuracy on the VoxCeleb1 identity classification task, surpassing high-profile models like MSM-MAE (95.3%), Audio-MAE (94.8%), and significantly outperforming wav2vec 2.0 (+20.9%) and HuBERT (+14.7%).

Comparison of SAIC with state-of-the-art methods on the VoxCeleb1 dataset.

Disentanglement and Synthesis

t-SNE visualizations confirm successful disentanglement: speaker embeddings cluster by individual, while content embeddings collapse into a single "blob," indicating identity information has been removed. Qualitative analysis of spectrograms shows that synthesized anonymized audio maintains high structural similarity to the original while adopting a new vocal identity.

Conclusion

SAIC provides a robust framework for simultaneous speech anonymization and identity classification. By leveraging CNNs and a two-stage training strategy, it achieves state-of-the-art results with greater efficiency than Transformer-based alternatives. Future work will focus on clinical validation using real-world medical recordings and generalizing the model for real-time healthcare monitoring.

Summary updated

BibTeX
@misc{cheng2023saic,
  author = {Ming Cheng and Xingjian Diao and Shitong Cheng and Wenjun Liu},
  title = {{SAIC: Integration of Speech Anonymization and Identity Classification}},
  year = {2023},
  eprint = {2312.15190},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2312.15190},
}
Figure