Abstract
Phone verification is a critical task in speech processing, traditionally handled by hybrid Neural Network/Hidden Markov Model (NN/HMM) systems. This paper introduces Hidden Logistic Linear Regression (HLLR), a method that extracts high-dimensional "Hidden Logistic Features" (HLFs) from neural network hidden layers instead of final posterior probabilities. By utilizing these features in a Support Vector Machine (SVM) classifier, we demonstrate that the normalization in the output layer discards discriminative information. Experimental results on the TIMIT database show that HLF-based systems, especially when using modular architectures and RBF kernels, achieve a record-low Equal Error Rate (EER) of 3.63%.
Introduction
In speech processing, phone verification determines if a speech segment matches a target phoneme—essentially detecting mispronunciations. Standard approaches utilize the posterior probabilities from a neural network's output layer. However, the authors hypothesize that the normalization required for these probabilities (ensuring they sum to 1) results in significant information loss. By the time data reaches the output layer, it has been "squashed" and compressed, losing the rich, high-dimensional nuances captured in earlier layers.
Methodology
The proposed Hidden Logistic Linear Regression (HLLR) framework shifts the focus from the output layer to the hidden layer activations.
Hidden Logistic Features (HLFs)
Instead of using Posterior Features (PF), the system extracts sigmoid activations from the hidden layer. These HLFs represent "latent attributes" of the speech. The first half of the neural network effectively acts as a non-linear kernel, transforming acoustic data into a high-dimensional space optimized for classification.
Modular Neural Networks (MNN)
To further enhance performance and training efficiency, the researchers implemented a modular approach. The input is split into "Left Context" (LC) and "Right Context" (RC) networks, which process preceding and following speech frames independently. This provides two distinct "views" of the target sound, providing a more robust feature set than a single monolithic network.
SVM Classification
The extracted HLFs are fed into an SVM back-end. The study evaluated Linear, Polynomial, and Radial Basis Function (RBF) kernels to determine the optimal separation boundary for correct pronunciations versus errors.
Experimental Setup
The study utilized the TIMIT acoustic-phonetic continuous speech corpus, a standard benchmark in the field. The training set comprised 462 speakers (3.12 hours of data), while the testing set included 168 speakers (1.14 hours). The researchers used a reduced set of 39 phones for the evaluation.
Key Results
The HLF approach consistently outperformed traditional PF methods across all tests:
- Standard FNN: Using HLFs reduced the Equal Error Rate (EER) from 4.28% to 4.13%.
- Modular Approach: Combining Left and Right HLFs (LC-HLF & RC-HLF) further dropped the EER to 3.88% using a linear SVM.
- Optimal Configuration: The best performance was achieved using the RBF kernel with combined modular HLFs, reaching an EER of 3.63%.
Beyond accuracy, the modular approach was significantly faster to train—taking about 2 hours per context network compared to 6 hours for the standard full-context network.
Discussion and Limitations
One unexpected finding was that while the Intermediary Neural Network (I-NN) improved general phone recognition, it performed poorly for verification (5.24% EER). The authors attribute this to the fact that averaging errors across different phones in the verification stage can mask the benefits seen in raw recognition accuracy.
Conclusion
Tapping into the hidden layers of neural networks provides a more robust feature set for phone verification than standard output probabilities. The HLLR method, combined with modular architectures and SVMs, sets a new performance benchmark on the TIMIT database. Future work will explore more complex feature fusion techniques and the impact of hidden layer quality over quantity.
Summary updated