Abstract
Standard speaker adaptation in Computer-Assisted Language Learning (CALL) often leads to "over-adaptation," where the system incorrectly validates a learner's pronunciation errors by warping the acoustic model to match them. This paper proposes Regularized-MLLR, which constrains adaptation to a mathematical space defined by native speakers. Experimental results demonstrate that this method prevents performance degradation as model complexity increases, significantly improving both pronunciation scoring correlation and error detection accuracy compared to standard Maximum Likelihood Linear Regression (MLLR).
Introduction
In CALL systems, speaker adaptation is essential to account for individual vocal characteristics like pitch and accent. The industry standard, Maximum Likelihood Linear Regression (MLLR), assumes the speaker is attempting "correct" pronunciation. However, for language learners, this assumption fails. If a student consistently mispronounces a phoneme, standard MLLR shifts the model's definition of that phoneme toward the error to maximize data likelihood. This "over-adaptation" effectively moves the goalposts, leading the system to judge errors as correct.
Proposed Method: Regularized-MLLR
To address over-adaptation, the authors introduce a regularization constraint on the transformation matrix. Instead of allowing the matrix to adapt freely to the learner's potentially erroneous speech, it is restricted to a "native manifold":
- Teacher Basis Construction: Transformation matrices are first calculated for a group of native English teachers, representing the space of "ideal" vocal variations.
- Linear Combination Constraint: The learner’s transformation matrix is forced to be a linear combination of these teacher matrices.
- Weight Optimization: The system solves for optimal weights ($\alpha$) that fit the learner's physical voice characteristics (e.g., gender, vocal tract length) while remaining within the space of native-like transformations.
By treating the learner's data only as a means to select the best mixture of "perfect" teacher shifts, the system avoids adapting to phonological errors that do not exist in the teacher space.
Experimental Evaluation
The method was evaluated using two datasets:
- ERJ (English Read by Japanese Students) Corpus: Used for automatic scoring, containing proficiency labels from experts for 42 learners.
- Japanese Learners of English Word Corpus: 5,950 utterances annotated with phonemic labels to test error detection.
Results and Discussion
A key finding is the relationship between model complexity (number of regression classes) and performance. In standard MLLR, increasing complexity typically worsens pronunciation evaluation because the model over-fits to errors. Regularized-MLLR remains stable or improves as complexity increases.
As shown in the figure above, Regularized-MLLR maintained a high correlation (~0.67) with human raters even as the number of nodes in the regression class tree increased to 64. In contrast, standard MLLR performance peaked early and then crashed as more classes were added, dropping below the baseline "no adaptation" level.
Furthermore, the proposed method significantly improved error detection metrics.
Regularized-MLLR achieved consistently higher precision and recall than standard MLLR. Notably, while standard MLLR's recall drops as the number of nodes increases, Regularized-MLLR remains stable, identifying more actual errors without sacrificing reliability.
Conclusion and Future Work
Regularized-MLLR effectively decouples physical speaker identity from phonological accuracy in adaptation. Future research will investigate the optimal number and variety of teachers needed for the basis, the minimum adaptation data required from learners, and whether specific phonemes (like liquids or fricatives) require specialized regularization strategies.
Summary updated