Abstract
Acoustic likelihood computations are one of the most computationally intensive parts of large vocabulary continuous speech recognition (LVCSR). In this paper, we introduce a method for accelerating these computations using graphical processing units (GPUs). By optimizing the evaluation process, we achieve an 11x speedup in acoustic probability evaluation.
Introduction
In Large Vocabulary Continuous Speech Recognition (LVCSR) systems, acoustic likelihood computation is a critical bottleneck. This process involves evaluating how well audio features match phonetic models, typically represented as Gaussian Mixture Models (GMMs). In traditional CPU-based decoders, this task can consume between 50% and 70% of the total decoding time. While libraries like Intel’s Integrated Performance Primitives (IPP) offer some relief, the increasing complexity of speech tasks necessitates more powerful parallel processing. This study explores mapping GMM calculations onto the parallel architecture of GPUs to overcome these performance hurdles.
Methodology
The research explores two primary strategies for GPU acceleration: Parallel Reduction and Matrix Multiplication.
Parallel Reduction
The first approach involves simplifying the GMM likelihood equation into a dot-product form. By pre-computing constants, the real-time workload is reduced to summing these values. A parallel reduction algorithm is employed to perform this summation, effectively reducing computational complexity from $O(n)$ to $O(\log_2 n)$. This method is particularly well-suited for the multi-core nature of GPUs.
Matrix Multiplication
To further optimize performance and leverage standard high-performance libraries, the authors transitioned to a matrix-based approach. GMM models and audio observations are stored as large matrices. Specifically, the observation vector is expanded to include squared terms and constants to facilitate a single matrix multiplication operation.
By utilizing NVIDIA’s cublasSgemm library, the system can compute likelihoods for multiple frames and states simultaneously. This approach masks memory latency more effectively than custom kernels. The final probability for each state is then calculated using a custom logAdd kernel.
Experimental Results
The performance was evaluated using two test sets: a small set (12 frames, 5,000 states) and a large set (181 frames, 5,786 states). Features consisted of 39-dimensional vectors, including MFCCs, energy, and their derivatives.
The GPU implementations were compared against a baseline using Intel IPP on a Xeon CPU:
- Parallel Reduction: Achieved a 3.8x speedup.
- Single-frame Matrix Multiplication: Achieved a 7.6x speedup.
- Multi-frame Matrix Multiplication: Achieved a peak speedup of 11.6x.
In practical terms, a task requiring 2.38 seconds on the CPU was reduced to 0.205 seconds on the GPU.
Discussion and Conclusion
While the GPU offers significant raw computational advantages, "Beam Pruning"—a technique used by CPUs to skip unlikely state calculations—remains a challenge. GPUs perform best with regular, non-branching data patterns, making the implementation of efficient pruning difficult. Consequently, the 11x computational speedup translates to approximately a 2x improvement in a full end-to-end decoder pipeline. Future work will likely focus on Deep Belief Networks (DBNs), which are inherently more compatible with GPU architectures than traditional GMMs.
Summary updated