Introduction
In Automatic Speech Recognition (ASR), models are typically trained on a wide variety of speakers to achieve "Speaker Independence" (SI). However, the unique vocal characteristics of individual users often lead to performance degradation. To mitigate this, researchers employ "Speaker Adaptation" to transform SI models into "Speaker Dependent" (SD) models.
Traditional adaptation techniques for Multi-Layer Perceptrons (MLPs) in hybrid HMM/ANN systems often involve adding auxiliary layers, such as a Linear Input Network (LIN) or Linear Hidden Network (LHN). While effective, these methods introduce architectural mismatches between SI and SD models and tie the number of adaptation parameters strictly to the dimensions of the input or output layers. This paper proposes a novel approach: adapting the shape of the hidden activation functions themselves using orthonormal Hermite polynomials.
The Hermitian-Based MLP (HMLP)
The core innovation of this research is the Hermitian-Based MLP (HMLP). Unlike standard neural networks that use fixed activation functions like Sigmoids or ReLUs, the HMLP utilizes a flexible function composed of a weighted sum of orthonormal Hermite polynomials.
The activation function $f_j(z)$ for a hidden unit $j$ is defined as: $$f_j(z) = \sum_{r=1}^{R} c_{jr} h_r(z)$$ where $h_r$ represents the Hermite functions and $c_{jr}$ are learnable coefficients. During the adaptation phase, the primary weights of the neural network remain frozen. Only the coefficients ($c_{jr}$) of the Hermite polynomials are updated via standard backpropagation. This strategy ensures that the network architecture remains identical between SI and SD versions while allowing for precise control over the number of adaptation parameters by adjusting the polynomial degree ($R$).
System Architecture and Feature Extraction
The HMLP is integrated into a standard hybrid HMM/ANN pipeline. The process begins with raw speech signals, which undergo feature extraction to produce Mel-bank outputs. These are then buffered, windowed, and transformed via Discrete Cosine Transforms (DCTs) before being fed into the MLP classifier.
Experimental Setup
The proposed system was evaluated using the Wall Street Journal (WSJ0) corpus:
- Training: The SI84 set, comprising approximately 15.3 hours of speech from 84 speakers.
- Adaptation: The
si_et_adset, featuring 8 speakers with 40 utterances each. - Evaluation: The Nov92 evaluation set, containing 330 utterances from 8 speakers.
Key Results
The HMM/HMLP system demonstrated significant improvements over standard baselines. By decoupling the adaptation parameter budget from the network geometry, the authors utilized only 24,000 parameters—far fewer than the 85,000 to 300,000 required by traditional LIN methods.
| System | SI WER (Baseline) | SD WER (Adapted) | Relative Improvement |
|---|---|---|---|
| HMM/HMLP (Proposed) | 6.4% | 5.5% | 14.1% |
| LOQ-1 (Standard) | 8.4% | 7.9% | 5.9% |
| LOQ-2 (Standard) | 6.5% | 5.6% | 13.8% |
The Hermitian approach achieved the lowest overall Word Error Rate (WER) of 5.5%, representing a 14.1% relative improvement over its SI baseline. This suggests that the shape of the non-linearity is a powerful lever for domain adaptation.
Conclusion and Future Work
This research highlights the efficacy of using flexible, Hermitian-based activation functions for speaker adaptation. By maintaining architectural consistency and providing a tunable parameter space, the HMLP offers a robust alternative to traditional linear network adaptation. Future work will explore scaling this approach to Large Vocabulary Continuous Speech Recognition (LVCSR) tasks, implementing conservative training techniques to prevent catastrophic forgetting, and applying these principles to deeper neural network architectures.
Summary updated