Abstract
This paper introduces Complementary Phone Error (CPE) training, a novel objective function for speech recognition designed to explicitly train acoustic models to produce errors distinct from existing systems. By incorporating a diversity term into the standard Minimum Phone Error (MPE) criterion, the method systematically generates complementary models for ensembling. Evaluation on large-scale Arabic broadcast transcription tasks demonstrates that combining these models via ROVER significantly reduces word error rates (WER), achieving an absolute improvement of up to 0.7% on a 172-hour development system.
Introduction
In high-performance Large Vocabulary Continuous Speech Recognition (LVCSR), the most robust results are typically achieved through system ensembling—combining the outputs of multiple models. The effectiveness of this approach relies heavily on the complementarity of the systems; they must fail on different words to allow a combination method like ROVER to select the correct hypothesis. Traditionally, diversity is introduced manually through varying acoustic features (e.g., PLP vs. MFCC) or neural architectures. This paper proposes a mathematical framework to force a new model to be accurate while ensuring its error patterns are distinct from a pre-existing baseline.
Methods
The researchers build upon the Minimum Phone Error (MPE) criterion, which adjusts model parameters to maximize expected phone accuracy against a ground-truth transcript. The proposed Complementary Phone Error (CPE) criterion modifies this objective to maximize a weighted combination of two factors:
- Accuracy: Expected phone accuracy relative to the ground-truth reference.
- Diversity: The negative expected phone accuracy relative to the output of a pre-existing "cross-system."
By subtracting accuracy relative to an existing system, the trainer incentivizes the model to be correct where the reference is correct, but to produce different sequences than the baseline when it deviates. A hyperparameter, $\alpha$, balances these two objectives. The researchers also explored an iterative approach, where a model trained via CPE is used as the diversity reference for a subsequent round of training.
Experimental Results
The Balancing Act ($\alpha$)
The choice of $\alpha$ is critical. If $\alpha$ is too high, individual model performance degrades significantly; if too low, the model simply replicates the baseline. As shown in the initial experiments, a "sweet spot" exists where the model maintains high accuracy while increasing disagreement with the baseline.
System Combination and Iterative Gains
The method was evaluated on an Arabic broadcast transcription task (DARPA GALE). On a 172-hour development system, the baseline MPE system achieved a WER of 21.6%. By combining the baseline with a second-iteration CPE model, the researchers achieved a significant reduction in error.
On the full 1831-hour Cambridge Arabic LVCSR setup, the gains were smaller but consistent, providing a 0.1% to 0.2% absolute reduction in WER. The iterative approach proved particularly effective, suggesting that pushing models away from the baseline in stages is more stable than a single aggressive push.
Conclusion
CPE training provides a systematic way to generate complementary models for speech recognition ensembles. By directly optimizing for diversity in the loss function, it removes the trial-and-error associated with manual feature engineering for ensembling. While the gains are more pronounced in smaller systems, the consistent improvements in large-scale tasks validate the approach. Future work may explore how this sequence-level diversity training translates to modern end-to-end deep learning architectures and more sophisticated combination logic beyond standard ROVER.
Summary updated