Abstract
Segmental Conditional Random Fields (SCRFs) offer a powerful framework for speech recognition by defining features over entire segments rather than individual frames. This paper systematically investigates various discriminative training criteria for SCRFs, including a novel "overlap cost" function. We evaluate standard log-loss against Empirical Bayes Risk (EBR), hinge loss, and ramp loss across two tasks: phonetic speech recognition (TIMIT) and ASL fingerspelling. Our findings indicate that while log-loss is a robust baseline, structured hinge loss combined with the proposed overlap cost and AdaGrad optimization provides superior accuracy and significantly faster convergence.
Introduction
Traditional Automatic Speech Recognition (ASR) systems typically utilize frame-by-frame modeling, such as Hidden Markov Models (HMMs). However, these models often suffer from conditional independence assumptions between frames and struggle to incorporate expressive features that span entire phonetic segments. Segmental models, particularly Segmental Conditional Random Fields (SCRFs), address these limitations by defining features over variable-length segments.
Historically, SCRFs have been trained using Conditional Likelihood (CL), or log-loss, which maximizes the probability of the correct sequence. While effective, CL does not explicitly minimize the final evaluation metric, such as Word Error Rate (WER). This study explores "cost-sensitive" training approaches that incorporate explicit error signals into the loss function to better align training objectives with performance goals.
Methods
We evaluate several loss functions and optimization strategies within a lattice-rescoring framework:
Loss Functions
The study compares standard Log-loss (CL) against three cost-sensitive alternatives:
- Empirical Bayes Risk (EBR): Minimizes the expected cost under the model's posterior.
- Hinge Loss: Used in Structured Support Vector Machines (SSVMs) to create a margin between the correct sequence and competitors.
- Ramp Loss: A non-convex loss that provides a tighter bound on the error than hinge loss.
The Overlap Cost
A key contribution of this work is the introduction of a novel Overlap Cost. Standard ASR training often employs the Minimum Phone Error (MPE) cost, which primarily penalizes false negatives. In contrast, the proposed Overlap Cost is based on the intersection-over-union of segments. By penalizing any portion of a hypothesized segment that does not overlap with the ground truth, it provides a more granular and informative error signal for the learning algorithm.
Optimization and Rescoring
To improve training efficiency, we compare AdaGrad against the traditionally used Rprop algorithm. Furthermore, to maintain computational tractability, the models perform lattice rescoring—evaluating a set of likely candidates generated by a baseline system rather than searching the entire hypothesis space.
Key Results
The experimental results demonstrate that the choice of both loss function and cost metric significantly impacts performance.
As shown in the figure above, Hinge loss combined with the novel overlap cost consistently emerged as the top-performing configuration across both tasks. While the tuned ramp loss achieved slightly lower error rates in the ASL task, its non-convex nature required extensive hyperparameter tuning, making hinge loss a more practical and robust choice.
One of the most significant findings was the effectiveness of the proposed overlap cost, which consistently outperformed the standard MPE cost across different loss functions. Additionally, the transition from Rprop to AdaGrad optimization resulted in a dramatic speedup, with AdaGrad often converging in a single epoch compared to the 4-5 epochs required for Rprop.
Discussion and Conclusion
This study highlights that while standard log-loss is a remarkably resilient baseline for segmental models, cost-sensitive training can yield meaningful improvements. The proposed overlap cost provides a superior error signal compared to traditional MPE costs. Furthermore, the use of AdaGrad significantly enhances the feasibility of training complex SCRFs.
Future research will investigate the application of these techniques to larger-scale tasks and explore "warm start" strategies to stabilize the optimization of non-convex losses like EBR and Ramp loss. The results also suggest that the importance of precise temporal boundaries in segmental modeling cannot be overstated, as even systems with high phone error rates can achieve competitive results if timing information is accurate.
Summary updated