TL;DR
This paper introduces SubDocTrans, a novel framework that significantly improves document-level machine translation (DocMT) by breaking down documents into smaller, manageable "sub-documents" and augmenting LLMs with multi-granularity knowledge. This approach tackles common issues like inconsistency and omission errors, leading to more coherent and accurate translations.
What Problem Does This Paper Address?
While Large Language Models (LLMs) have revolutionized sentence-level translation, translating entire documents (DocMT) remains a significant challenge. LLMs often struggle with maintaining context and coherence across multiple sentences and paragraphs, leading to translations that can be inconsistent, lack flow, or even omit crucial information. Existing approaches either process documents as a whole (which can be computationally expensive and lead to errors) or break them down into individual sentences (losing valuable inter-sentence context). This paper aims to bridge this gap by developing a method that effectively leverages document-level context without sacrificing efficiency or accuracy.
Methods
SubDocTrans employs a three-step approach:
- Document Segmentation: The core innovation is dividing a long document into smaller, overlapping "sub-documents" based on topic coherence. This segmentation allows for more focused processing while preserving inter-paragraph flow.
- Multi-granularity Knowledge Extraction: For each sub-document, the framework extracts various types of knowledge:
- Global Knowledge: Bilingual summaries and themes that capture the overarching context of the entire document.
- Local Knowledge: Topics specific to each sub-document and transition hints that describe how one sub-document logically follows the previous one.
-
Proper Nouns: Extracted and translated to ensure terminological consistency throughout the document. This extracted knowledge is then integrated into LLM prompts.
-
Knowledge-Augmented Translation: The LLM is prompted with the sub-document and the extracted knowledge to generate a translation. A "sentence alignment strategy" is also employed to ensure a one-to-one mapping between source and target sentences, further reducing omission errors.
Datasets
The authors evaluated SubDocTrans on two datasets:
- IWSLT2017: A standard dataset for machine translation, comprising parallel documents from TED talks.
- Guofeng Web-novel: A high-quality, discourse-level web novel corpus, which presents a more challenging test for document-level translation.
Key Insights and Ideas
- Sub-document segmentation is key: Breaking down long documents into coherent sub-documents is a surprisingly effective way to manage context and improve translation quality without overwhelming the LLM.
- Multi-granularity knowledge is powerful: Combining global (summary, theme) and local (topics, transition hints) knowledge provides a richer context for the LLM, leading to more consistent and coherent translations.
- Proper noun consistency matters: Explicitly extracting and translating proper nouns helps maintain terminological accuracy across the entire document, a common pitfall in DocMT.
- Sentence alignment strategy reduces errors: The simple yet effective sentence alignment strategy directly addresses omission errors, a persistent problem in previous DocMT approaches.
Key Results
SubDocTrans consistently outperformed advanced baselines across various metrics and translation directions on both the IWSLT2017 and Guofeng datasets.
- Translation Quality & Consistency: SubDocTrans achieved the highest scores in sCOMET (a sentence-level quality metric) and LTCR-1 (a consistency metric), often surpassing strong baselines like DELTA by significant margins (e.g., 1.49 points in sCOMET and 1.2 points in LTCR-1).
- Discourse Phenomena: The framework showed the best performance in handling discourse phenomena, including tense consistency, conjunction presence, and pronoun translation accuracy.
- Omission Errors & Hallucinations: SubDocTrans significantly reduced omission errors (0.75% vs. 8.75% for Doc2Doc) and showed promising results in mitigating hallucinations.
- Inference Efficiency: While not the absolute fastest (due to LLM calls for knowledge extraction), SubDocTrans offered a good trade-off between efficiency and high-quality translation, completing translations in a reasonable time with minimal retries.
Limitations and Future Work
The authors acknowledge a few limitations:
- Computational Overhead: The knowledge extraction process, particularly relying on LLMs, can introduce computational overhead, potentially limiting real-time applications.
- LLM Efficiency for Extraction: Using LLMs for extracting proper nouns, while flexible, might be less efficient than specialized rule-based methods.
Future work could explore more efficient extraction techniques and investigate how to further optimize the computational cost of the framework.
Summary updated