Introduction
Traditional speaker verification systems have long relied on hand-crafted acoustic features such as Mel-frequency cepstral coefficients (MFCCs) or spectrograms. While effective, these features require manual hyperparameter tuning and assume a fixed linear or logarithmic frequency scale. RawNet addresses these limitations by processing raw audio waveforms directly, allowing the model to learn task-specific feature extractions in an end-to-end fashion.
Architecture and Methodology
The RawNet architecture is divided into a feature-extracting front-end and a classification back-end.
Front-end Feature Extraction
The front-end begins with a strided convolutional layer that processes raw time-domain signals. This is followed by a series of residual blocks (CNN) designed to capture frame-level characteristics. These features are then aggregated into a single utterance-level speaker embedding using a Gated Recurrent Unit (GRU).
To optimize the embedding space, the authors employ a multi-objective loss function combining Categorical Cross-Entropy, Center Loss (to minimize intra-class variance), and Speaker Basis Loss (to maximize inter-class variance).
Back-end Classification
A novel "concat&mul" DNN classifier is introduced for the verification stage. When comparing two embeddings, the system concatenates the vectors and includes their element-wise product. This combined representation is processed by a four-layer fully connected network to determine speaker identity.
Experimental Setup
The system was evaluated using the VoxCeleb1 dataset, which contains over 330 hours of "in-the-wild" audio from 1,251 speakers. The standard split was used: 1,211 speakers for training and 40 for evaluation.
Results and Discussion
RawNet demonstrates superior performance compared to traditional baseline systems. As shown in the comparison below, RawNet achieves a significantly lower Equal Error Rate (EER) than standard i-vector and x-vector implementations when data augmentation is not utilized.
Specifically, RawNet reached an EER of 4.8% using simple cosine similarity. By implementing the proposed "concat&mul" back-end, the error rate was further reduced to 4.0%. Notably, RawNet remains competitive with state-of-the-art x-vector systems even without the intensive data augmentation (noise and reverberation) typically required by those models.
Conclusion
RawNet successfully eliminates the need for manual feature engineering in speaker verification. By learning frequency responses directly from raw waveforms and utilizing a robust multi-objective loss, it sets a new benchmark for non-augmented end-to-end systems. Future research will focus on integrating data augmentation and exploring the model's performance on shorter utterances and diverse languages.
Summary updated