Isometric Neural Machine Translation using Phoneme Count Ratio Reward-based Reinforcement Learning

Shivam Mhaskar, Nirmesh Shah, Mohammadi Zaki, Ashishkumar P. Gudmalwar, Pankaj Wasnik, Rajiv Ratn ShahNAACL 2024 · DOI

Abstract

Achieving "isochrony"—matching the duration of translated speech to the source—is a critical challenge in Automatic Video Dubbing (AVD). This paper introduces a Reinforcement Learning (RL) framework for Isometric Neural Machine Translation (NMT) that utilizes Phoneme Count Ratios (PCR) as a reward signal. To mitigate the typical quality degradation associated with strict length constraints, we propose a Student-Teacher architecture (ST-RL-NMT) that leverages Knowledge Distillation. Our approach demonstrates a 36% improvement in length compliance for English-to-Hindi translation while preserving linguistic accuracy, outperforming both standard NMT baselines and general-purpose Large Language Models.

Introduction

The immersion of dubbed video content relies heavily on the synchronization of audio with visual cues, such as lip movements and scene transitions. Standard NMT models often produce translations of varying lengths, leading to "desynchronization" where the dubbed audio is either too long or too short for the original video segment. While previous attempts at Isometric NMT focused on word or character counts, these metrics are poor proxies for actual speech duration. This work argues that phonemes—the fundamental units of sound—provide a more accurate basis for matching speech timing across languages, particularly for Indian languages like Hindi where syllable mapping is complex.

Methodology

The proposed solution consists of two primary components: a phoneme-based RL loop and a stabilizing Student-Teacher framework.

RL-NMT: Phoneme-Based Rewards

We frame the translation task as an RL problem where the NMT model acts as an agent. The reward function is based on the Phoneme Count Ratio (PCR), defined as the ratio of phonemes in the generated translation to those in the source text. Translations falling within a target window (e.g., $1 \pm \delta_i$) receive a positive reward and are used for iterative fine-tuning. As training progresses, the threshold $\delta_i$ is tightened to enforce stricter isochrony.

ST-RL-NMT: Preserving Quality via Knowledge Distillation

Enforcing strict length constraints often forces models to omit critical information or produce ungrammatical output. To counter this, we employ a Student-Teacher architecture. A high-quality, unconstrained NMT model serves as the "Teacher," providing linguistic guidance to the "Student" (the RL-tuned model). By minimizing the KL Divergence between the Student's and Teacher's output distributions, the model learns to meet length targets without sacrificing translation quality.

Architectural overview of the proposed RL-NMT (a) and the modified ST-RL-NMT agent (b).

Experimental Results

The models were evaluated on the Bharat Parallel Corpus Collection (BPCC), Flores, and specialized Movie Subtitles datasets. We introduced the Phoneme Count Compliance (PCC) score to measure the percentage of translations meeting specific length thresholds.

Key findings include:

  • Significant Compliance Gains: The ST-RL-NMT approach improved PCC scores by approximately 36% over state-of-the-art English-Hindi baselines.
  • Effective Quality-Length Trade-off: As shown in the performance plots for the Movie test set, while pure RL can lead to a decline in BLEU scores, the Student-Teacher framework stabilizes the model. The final model achieved a BLEU score of 37.70, nearly matching the baseline (38.41) while significantly increasing length compliance.
  • Superiority over LLMs: The specialized isometric models outperformed general-purpose models like LLaMA2-7B in both translation accuracy and length control for this specific task.
Performance metrics across RL iterations for the Movie test set, showing the evolution of BLEU, CHRF, PCC, and BLEURT scores.

Conclusion

By shifting the focus from character counts to phoneme counts and utilizing a Student-Teacher RL framework, we have developed a robust method for Isometric NMT. This approach significantly enhances the feasibility of high-quality, synchronized automatic dubbing. Future research will explore soft-threshold reward functions and the generalization of phoneme-based rewards to a broader range of language families.

Summary updated

BibTeX
@inproceedings{mhaskar2024isometric,
  author = {Shivam Mhaskar and Nirmesh Shah and Mohammadi Zaki and Ashishkumar P. Gudmalwar and Pankaj Wasnik and Rajiv Ratn Shah},
  title = {{Isometric Neural Machine Translation using Phoneme Count Ratio Reward-based Reinforcement Learning}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2024},
  doi = {10.18653/v1/2024.findings-naacl.250},
}
Figure