RawNet: Advanced End-to-End Deep Neural Network Using Raw Waveforms for Text-Independent Speaker Verification

Jee-weon Jung, Hee-Soo Heo, Ju-ho Kim, Hye-jin Shim, Ha-Jin YuInterspeech 2019 · DOI

Introduction

Traditional speaker verification systems have long relied on hand-crafted acoustic features such as Mel-frequency cepstral coefficients (MFCCs) or spectrograms. While effective, these features require manual hyperparameter tuning and assume a fixed linear or logarithmic frequency scale. RawNet addresses these limitations by processing raw audio waveforms directly, allowing the model to learn task-specific feature extractions in an end-to-end fashion.

Architecture and Methodology

The RawNet architecture is divided into a feature-extracting front-end and a classification back-end.

Front-end Feature Extraction

The front-end begins with a strided convolutional layer that processes raw time-domain signals. This is followed by a series of residual blocks (CNN) designed to capture frame-level characteristics. These features are then aggregated into a single utterance-level speaker embedding using a Gated Recurrent Unit (GRU).

The RawNet architecture, illustrating the flow from raw input samples to the final output layer.

To optimize the embedding space, the authors employ a multi-objective loss function combining Categorical Cross-Entropy, Center Loss (to minimize intra-class variance), and Speaker Basis Loss (to maximize inter-class variance).

Back-end Classification

A novel "concat&mul" DNN classifier is introduced for the verification stage. When comparing two embeddings, the system concatenates the vectors and includes their element-wise product. This combined representation is processed by a four-layer fully connected network to determine speaker identity.

Experimental Setup

The system was evaluated using the VoxCeleb1 dataset, which contains over 330 hours of "in-the-wild" audio from 1,251 speakers. The standard split was used: 1,211 speakers for training and 40 for evaluation.

Results and Discussion

RawNet demonstrates superior performance compared to traditional baseline systems. As shown in the comparison below, RawNet achieves a significantly lower Equal Error Rate (EER) than standard i-vector and x-vector implementations when data augmentation is not utilized.

Comparison of Equal Error Rate (EER) between RawNet and baseline systems on the VoxCeleb1 dataset.

Specifically, RawNet reached an EER of 4.8% using simple cosine similarity. By implementing the proposed "concat&mul" back-end, the error rate was further reduced to 4.0%. Notably, RawNet remains competitive with state-of-the-art x-vector systems even without the intensive data augmentation (noise and reverberation) typically required by those models.

Conclusion

RawNet successfully eliminates the need for manual feature engineering in speaker verification. By learning frequency responses directly from raw waveforms and utilizing a robust multi-objective loss, it sets a new benchmark for non-augmented end-to-end systems. Future research will focus on integrating data augmentation and exploring the model's performance on shorter utterances and diverse languages.

Summary updated

BibTeX
@inproceedings{jung2019rawnet,
  author = {Jee-weon Jung and Hee-Soo Heo and Ju-ho Kim and Hye-jin Shim and Ha-Jin Yu},
  title = {{RawNet: Advanced End-to-End Deep Neural Network Using Raw Waveforms for Text-Independent Speaker Verification}},
  booktitle = {Interspeech},
  year = {2019},
  doi = {10.21437/interspeech.2019-1982},
}
Figure