Abstract
Translating lyrics for musicals presents unique challenges due to the need to ensure high translation quality while adhering to singability requirements such as length and rhyme. Existing song translation approaches often prioritize these singability constraints at the expense of translation quality, which is crucial for musicals. This paper aims to enhance translation quality while maintaining key singability features. Our method consists of three main components. First, we create a dataset to train reward models for the automatic evaluation of translation quality. Second, to enhance both singability and translation quality, we implement a two-stage training process with filtering techniques. Finally, we introduce an inference-time optimization framework for translating entire songs. Extensive experiments, including both automatic and human evaluations, demonstrate significant improvements over baseline methods and validate the effectiveness of each component in our approach.
Introduction
Translating lyrics for musicals is a complex balancing act known as the "Pentathlon Principle," requiring harmony between singability, rhyme, rhythm, sense, and naturalness. In musical theater, lyrics are not just songs but narrative vehicles; a poor translation can obscure plot points or character development.
Current Automatic Song Translation (AST) systems often over-prioritize technical constraints like syllable counts and rhyme schemes. This frequently leads to "translationese"—text that technically fits the music but lacks linguistic grace or narrative depth. As shown in Figure 1, standard translation tools often fail to match the required length or rhyme, while specialized baselines may produce awkward phrasing to meet musical constraints.
Methodology
To address these challenges, the researchers proposed a three-pronged framework designed to elevate the literary quality of translated lyrics without sacrificing singability.
1. Reward Model Training
The team developed two specialized reward models: $R_{bas}$ for basic fluency and accuracy, and $R_{adv}$ for aesthetic "literacy" or literary merit. These models were trained on MusicalTransEval, a new dataset of 3,938 high-quality English-Chinese lyric pairs from 11 diverse musicals, labeled by humans for linguistic sophistication.
2. Two-Stage Training Process
Rather than training on a massive, noisy dataset, the model undergoes a refined two-stage process:
- Stage 1: The model learns basic singability (length and rhyme) using a large-scale song dataset of 1.75M pairs.
- Stage 2: The model is fine-tuned on a "High-Quality" subset (700K pairs) filtered by the reward models to refine its literary style and narrative accuracy.
3. Inference-Time Optimization
Translating a song line-by-line often results in inconsistent rhyming. The authors introduced an optimization framework that generates multiple candidates for each line and uses a paragraph-level loss function to select the best combination, balancing length accuracy, rhyme consistency, and reward model scores.
Key Results and Insights
The proposed model, Ours VER.3, significantly outperformed previous state-of-the-art methods across both automatic and human metrics:
- Translation Fidelity: Achieved a COMET score of 70.57, compared to 67.95 for the baseline, indicating much higher fidelity to the original meaning.
- Human Evaluation: In blind tests, human evaluators rated the model higher in Fluency (3.29/4), Accuracy (2.89/4), and Literacy (2.67/4).
- Singability: The model maintained a high Length Accuracy of 99.2%. Unlike baselines that force rhymes at the cost of sentence structure, this approach optimizes for rhyme at the paragraph level, resulting in more natural and performable lyrics.
One of the key findings was that even advanced models like GPT-4o struggle with these constraints in a zero-shot setting, often achieving only ~30% length accuracy. This underscores the necessity of the specialized fine-tuning and filtering techniques introduced in this work.
Conclusion and Future Work
This research demonstrates that for the specific needs of musical theater, quality-oriented filtering and paragraph-level optimization are more effective than simply increasing training data size. While the current model excels at length and rhyme, future iterations could incorporate more professional musical constraints, such as word stress, vowel quality for high notes, and breath points. Expanding this framework to other language pairs also remains a priority for making global musical theater more accessible.
Summary updated