Abstract
Literary translation, particularly for long-form web novels, presents unique challenges in maintaining discourse-level coherence and terminology consistency. The SJTULoveFiction team addressed these issues for the WMT24 Discourse-Level Literary Translation task using a novel multi-agent pipeline. Their approach combines a chunk-based processing strategy with a multi-model ensemble, refined by specialized Translation Editor and Term Proofreader agents. Experimental results demonstrate that this collaborative agent framework significantly outperforms individual state-of-the-art models in both linguistic quality and narrative consistency.
Introduction
Traditional Machine Translation (MT) systems often struggle with the nuances of literature. Unlike technical or news text, literary content relies heavily on discourse-level coherence, where sentences must flow naturally over long spans. Furthermore, maintaining terminology consistency—ensuring a character's name or a specific world-building element remains identical throughout a chapter—is critical for reader immersion. Standard sentence-level MT often loses this "big picture" context. This paper details a system designed to leverage the massive context windows and reasoning capabilities of Large Language Models (LLMs) to preserve the narrative thread in long-form translations.
Methodology
The SJTULoveFiction system utilizes a sophisticated four-step pipeline designed to move beyond simple sequence-to-sequence translation.
1. Chunk-Based Strategy
To provide models with sufficient context, the team implemented a chunking strategy. Instead of translating individual sentences, text is grouped into segments ranging from 5 to 80 sentences. This allows the LLMs to understand surrounding stylistic and grammatical cues, leading to more cohesive output.
2. Multi-Model Translation and SFT
The system employs an ensemble of three top-tier models: a fine-tuned Qwen2-72B (trained on the specialized GuoFeng Webnovel Corpus), Claude-3.5, and GPT-4o. This diversity ensures a broad range of stylistic candidates for the subsequent refinement stages.
3. Translation Editor Agent
Recognizing that different models possess varying strengths, the team deployed GPT-4o as a "Translation Editor." This agent evaluates the source text alongside the three candidate translations, merging the most accurate and stylistically appropriate segments into a single, optimized version.
4. Terminology Intervention
To anchor the translation, a Term Proofreader Agent was developed. This agent extracts key terms from the merged translation, facilitates the selection of an "optimal" translation for each term, and then performs a global sweep to enforce these terms consistently across the entire chapter.
Key Insights and Results
The "Goldilocks" Chunk Size
Research into context windows revealed that more context is not always better. The team identified a "sweet spot" for chunk sizes; while training on 10-sentence chunks was effective for the fine-tuned Qwen model, inference on 5-sentence chunks often provided the best balance of context and accuracy for general-purpose models like Claude and GPT-4o.
Superiority of Merging
The translation merging strategy proved highly effective. In Linguistic Quality Ratings (LQR), the merged output achieved a "Completely correct and fluent" (LQR5) score 24% of the time, a significant improvement over the 9% achieved by GPT-4o acting alone.
Consistency Gains
The impact of the Term Proofreader Agent was particularly striking. Before intervention, a single Chinese term often resulted in an average of 3.73 different English variations throughout a text. Following the agent's intervention, this variance dropped to 1.12, dramatically improving the narrative's internal consistency.
Conclusion and Future Work
The SJTULoveFiction system demonstrates that the future of literary MT lies in refinement rather than just generation. By treating LLMs as evaluators and editors within a multi-agent framework, the system successfully synthesizes the creative strengths of multiple models while enforcing strict terminology constraints. Future work will focus on optimizing techniques for ultra-long context spans and automating the "optimal" term selection process to further reduce the need for human-in-the-loop intervention.
Summary updated