TL;DR
This paper explores novel neural network (NN) architectures for extracting probabilistic features to improve Large Vocabulary Continuous Speech Recognition (LVCSR). By combining hierarchical processing and "bottle neck" architectures, they achieve significant relative improvements over baseline MFCC systems and existing NN feature extraction methods.
What Problem Does This Paper Address?
Traditional LVCSR systems heavily rely on Mel-Frequency Cepstral Coefficients (MFCCs) as acoustic features. While effective, these short-term features can miss crucial temporal context. Researchers have explored using neural networks to extract more informative probabilistic features, often by mapping spectral information to phoneme posterior probabilities. However, the optimal NN architecture and input features for this task remain an open question. This paper investigates two promising NN topologies – hierarchical processing and "bottle neck" features – and their combination, aiming to leverage the strengths of both for enhanced speech recognition performance.
Methods
The authors explore two main NN topologies for probabilistic feature extraction:
- Hierarchical Processing: This approach uses a cascade of NNs. A subsequent NN takes the probabilistic features (outputs) from a preceding NN as its input, allowing for a multi-stage refinement of features. This is illustrated in Figure 1.
- Bottle Neck Features: This topology involves training a deeper NN (e.g., 5 layers) but extracting features from an intermediate "bottle neck" layer. This layer is designed to be smaller than the input or output layers, forcing the network to learn a compressed, yet discriminative, representation of the input. A key advantage is that the output feature size is independent of the number of phonetic targets, as described in Figure 2.
They also investigate two types of input features for these NNs:
- TRAP-DCT Features: These are based on temporal trajectories of energies in critical bands, processed with a Discrete Cosine Transform (DCT) to capture long-term spectral patterns.
- Multi-resolution RASTA (MRASTA) Features: An extension of RASTA filtering, these use multi-dimensional band-pass filters to capture different ranges of modulation frequencies, offering a multi-resolution spectral analysis.
The paper proposes and evaluates a Hierarchical Bottle Neck Processing approach, which combines the strengths of both topologies.
Datasets
The experiments were conducted using American Broadcast News (BN) data, totaling approximately 310 hours. This data was split into:
- Training: 140 hours of HUB4 speech data and 170 hours of TDT4 speech data.
- Development/Tuning: Broadcast News Transcription development corpus (DEV04) and DEV09 from the QUAERO 2009 Evaluation.
- Testing: Broadcast News evaluation corpora EVAL03, EVAL02, and EVAL09 from the QUAERO 2009 Evaluation.
The QUAERO corpora also included speech from European Parliamentary Plenary Sessions (EPPS) and web data. A 4-gram language model was trained on English Gigaword, HUB4, and TDT4 data.
Key Insights and Ideas
- Synergy of Hierarchical and Bottle Neck: Neither hierarchical processing nor bottle neck features alone consistently outperformed the other. However, their combination (Hierarchical Bottle Neck Processing) yielded the best results, suggesting a powerful synergy.
- Input Feature Importance: TRAP-DCT features generally performed better than MRASTA features when used with the bottle neck architecture, although MRASTA features showed better generalization on some corpora.
- Significant Improvement over Baseline: The proposed NN-based features, especially the Hierarchical Bottle Neck approach, significantly improved upon a baseline MFCC-only system, demonstrating the value of probabilistic features.
- Flexibility of Bottle Neck: The bottle neck architecture's ability to produce fixed-size features independent of the number of output classes is a valuable design choice.
Key Results
- The proposed Hierarchical Bottle Neck Processing system improved the overall LVCSR system by 3%-7% relative compared to systems using only hierarchical or only bottle neck features.
- The single MFCC baseline system was improved by up to 12% relative for the QUAERO corpora and 17%-23% relative for other corpora when augmented with the best NN features.
- Specifically, the HIER-BNECK.MRASTA system achieved the best results, reducing the error rate on Broadcast News DEV04 from 27.2% (MFCC only) to 22.5%.
Limitations and Future Work
- Further Input Feature Analysis: While TRAP-DCT showed better performance in some cases, MRASTA features demonstrated better generalization on certain corpora. Further investigation is needed to fully understand the performance differences.
- Splitting TRAP-DCT Features: The authors plan to split TRAP-DCT features, similar to how MRASTA features were processed, to potentially extract more gain from the Hierarchical Bottle Neck approach.
- Speaker Adaptation and Discriminative Training: The current experiments were performed without speaker adaptation or discriminative training. The authors intend to incorporate these techniques in future work, expecting further improvements.
- Limited to BN Data: While the results are promising, they are primarily demonstrated on Broadcast News data. Evaluating on a wider variety of speech data would be beneficial.
Summary updated