Introduction
Modern speech recognition systems are complex pipelines involving multiple decoding passes, model adaptations, and rescoring stages. These systems rely on numerous tunable parameters, such as pruning thresholds and model weights, which define the critical balance between decoding speed and accuracy. Traditionally, these parameters are manually tuned—a process often described as "black magic" that fails to scale across different languages or specialized tasks like Keyword Search (KWS). As data volumes increase and real-time requirements tighten, automated, metric-agnostic optimization under hard speed constraints has become essential.
System Architecture and Methods
The researchers utilized BBN’s Byblos sub-real-time speech recognizer integrated with a Keyword Search (KWS) component. The recognizer employs a multi-pass approach: a "fast match" forward pass using a simplified acoustic model (STM AM) and 2-gram language model (LM), followed by a "beam search" backward pass using a more complex model (SCTM AM) and 3-gram LM. The output is a pruned word lattice converted into a consensus network for indexing and search.
To optimize this system, the authors proposed an iterative algorithm based on the empirical observation that, for small perturbations, the effects of individual parameters on speed and accuracy are approximately linear and additive. The optimization process involves:
- Initialization: Starting with default parameters and defining small perturbation steps ($\Delta$) for each.
- Exploration: Running parallel decodes for $2N+1$ configurations (where $N$ is the number of parameters) to test the impact of nudging each parameter up or down.
- Vector Update: Treating results as vectors in a 2D space (Speed vs. Accuracy) and calculating a weighted combination to move toward the target speed. If the system is too slow, it prioritizes parameters that reduce time with minimal accuracy cost; if faster than the target, it boosts accuracy with minimal speed loss.
- Iteration: Repeating the process until the target speed is met and accuracy plateaus.
Experimental Setup
The approach was evaluated using 250 hours of Spanish training data (Fisher, CallHome, CallFriend, and Ricardo collections) and the 3.75-hour Spanish CallHome Eval97 test set. The primary task was Keyword Search, where performance is measured by the Area Under the Curve (AUC).
Key Results
The optimization algorithm was applied to 11 key parameters within the Byblos recognizer. The system initially operated at a decoding speed of 0.113xRT (approximately 8.8x faster than real-time).
As shown in the figure above, the algorithm successfully guided the system toward the target speed of 0.067xRT (15x faster than real-time) over 8 iterations. Remarkably, this significant speedup resulted in negligible accuracy loss, with the KWS AUC only decreasing from 0.4501 to 0.448. The automated process reached this optimal configuration in a fraction of the time required for manual tuning, with each iteration taking approximately one hour of wall-clock time due to parallel execution.
Insights and Future Work
The study highlighted a functional division in two-pass decoders: "fast match" parameters primarily drive speed improvements, while "beam search" parameters are more effective for fine-tuning accuracy. The "objective function blind" nature of the algorithm allows it to optimize for any measurable metric, including AUC or Topic ID, rather than being limited to Word Error Rate (WER).
Future research will address parameter dependencies, as the current model assumes independence. Implementing dynamic step sizes ($\Delta$) could further accelerate convergence. Additionally, while the linear assumption holds for small intervals, bridging large speed gaps may require intermediate targets to maintain stability. Finally, mitigating hardware-related noise in speed measurements remains a practical challenge for scaling these optimizations on shared computing clusters.
Summary updated