GRAFT: A Graph-based Flow-aware Agentic Framework for Document-level Machine Translation

Himanshu Dutta, Sunny Manchanda, Bapat, Prakhar, Meva Ram Gurjar, Pushpak BhattacharyyaarXiv 2025 · arXiv

TL;DR

This paper introduces GRAFT, a novel agent-based framework for document-level machine translation that uses Large Language Models (LLMs) to represent documents as directed acyclic graphs (DAGs). GRAFT significantly improves translation quality by explicitly modeling discourse-level phenomena, leading to more coherent and contextually accurate translations compared to existing state-of-the-art methods.

What Problem Does This Paper Address?

Document-level machine translation (DocMT) aims to translate not just individual sentences but entire documents while maintaining coherence, consistency, and capturing discourse-level phenomena. Existing DocMT approaches often struggle with this because they either use heuristic rules for segmenting documents into discourse units that don't align with true discourse structure, or they fail to maintain consistency across the document. This leads to issues like incorrect pronoun resolution, inconsistent terminology, and a lack of smooth transitions between sentences and paragraphs. The paper addresses the gap in effectively capturing and leveraging discourse structure for more accurate and coherent document translation.

Methods

GRAFT employs a multi-agent system, where each agent is powered by an LLM, to tackle document-level translation. The core of the framework is the transformation of a source document into a Directed Acyclic Graph (DAG). This is achieved through four specialized agents:

  • Discourse Agent: Segments the document into self-contained discourse units, treating each as a node in the DAG. This agent uses an LLM with a few-shot prompting strategy to decide sentence boundaries.
  • Edge Agent: Establishes directed edges between discourse nodes in the DAG, representing dependencies that require information sharing for translation. This agent also uses an LLM with few-shot prompting to determine these connections.
  • Memory Agent: Extracts structured "local memory" from each translated discourse. This memory includes mappings for pronouns, named entities, phrase-to-phrase mappings, discourse connectives, and a one-line summary of special translation decisions.
  • Translation Agent: Combines each discourse with its incident memories (prioritizing earlier dependencies) and uses an LLM to produce contextually informed translations.

This agentic pipeline iteratively processes discourses, ensuring that context from previous segments is propagated to inform the translation of subsequent ones.

Datasets

The paper uses several datasets for training and evaluation:

  • mZPRT: For Chinese-English translation, focusing on fiction and Q&A domains.
  • WMT2022: For Chinese-English translation in news and social domains.
  • Guofeng V1 TEST_2: For Chinese-English discourse-level literary translation (web fiction).
  • TED tst2017 (from IWSLT2017): For English-Chinese, English-French, English-German, and English-Japanese translation.

Key Insights and Ideas

  • DAG Representation is Crucial: Representing a document as a DAG effectively models the complex dependencies between discourse units, which is essential for capturing inter-discourse phenomena.
  • LLM Agents for Specialized Tasks: Decomposing the DocMT task into specialized subtasks handled by LLM agents (Discourse, Edge, Memory, Translation) allows for more nuanced and effective processing of discourse-level information.
  • Structured Memory Enhances Coherence: The Memory Agent's ability to extract and utilize structured local memory (pronoun mappings, entity mappings, etc.) significantly improves translation consistency and coherence.
  • Balancing Granularity and Cohesion: The Discourse Agent's segmentation strategy aims to create units that are internally coherent while still allowing for inter-discourse connections, striking a balance between granularity and cohesion.
  • Efficiency with Smaller LLMs: GRAFT demonstrates competitive performance even with smaller LLM backbones, highlighting the efficacy of its agentic and graph-based approach.

Key Results

  • Significant Performance Gains: GRAFT achieved an average improvement of 2.8 d-BLEU on TED test sets from IWSLT2017 and 2.3 d-BLEU for domain-specific English-to-Chinese translation over strong baselines.
  • Superior Discourse Handling: Analyses highlighted GRAFT's consistent ability to address discourse-level phenomena, yielding coherent and contextually accurate translations.
  • Discourse Segmentation Effectiveness: The LLM-based Discourse Agent outperformed random and semantic similarity-based segmentation strategies by an average of 11.3 and 3.6 d-BLEU, respectively.
  • Dependency Modelling Advantage: The LLM-based Edge Agent surpassed chain graph and TF-IDF based dependency modeling strategies, yielding an average gain of 2.0 d-BLEU.
  • Domain-Specific Performance: In domain-specific translation (Zh⇒En), GRAFT achieved an average improvement of 2.0 d-BLEU over GPT-4o-mini and 7.5 d-BLEU over GPT-3.5, demonstrating superior handling of terminology and pronoun translation.
  • Human Evaluation: Human evaluations showed high scores for GRAFT in general quality and discourse awareness, with scores of 4.1/4.2 for Zh⇒En translation.

Limitations and Future Work

  • Computational Overhead: The multi-agent design of GRAFT introduces computational overhead and higher latency compared to monolithic systems, as it involves multiple LLM calls.
  • Hyperparameter Sensitivity: Performance can be sensitive to hyperparameters like memory size, which may require domain-specific tuning.
  • Low-Resource Domains: The effectiveness of GRAFT on low-resource domains remains to be explored.

Future work could focus on:

  • Exploring additional applications of memory mechanisms to strengthen cross-discourse coherence.
  • Optimizing GRAFT's architecture to reduce latency and cost, potentially through caching or fine-tuning specific components.
  • Further enhancing dependency modeling.

Summary updated

Abstract

Document level Machine Translation (DocMT) approaches often struggle with effectively capturing discourse level phenomena. Existing approaches rely on heuristic rules to segment documents into discourse units, which rarely align with the true discourse structure required for accurate translation. Otherwise, they fail to maintain consistency throughout the document during translation. To address these challenges, we propose Graph Augmented Agentic Framework for Document Level Translation (GRAFT), a novel graph based DocMT system that leverages Large Language Model (LLM) agents for document translation. Our approach integrates segmentation, directed acyclic graph (DAG) based dependency modelling, and discourse aware translation into a cohesive framework. Experiments conducted across eight translation directions and six diverse domains demonstrate that GRAFT achieves significant performance gains over state of the art DocMT systems. Specifically, GRAFT delivers an average improvement of 2.8 d BLEU on the TED test sets from IWSLT2017 over strong baselines and 2.3 d BLEU for domain specific translation from English to Chinese. Moreover, our analyses highlight the consistent ability of GRAFT to address discourse level phenomena, yielding coherent and contextually accurate translations.
BibTeX
@misc{dutta2025graft,
  author = {Himanshu Dutta and Sunny Manchanda and Bapat, Prakhar and Meva Ram Gurjar and Pushpak Bhattacharyya},
  title = {{GRAFT: A Graph-based Flow-aware Agentic Framework for Document-level Machine Translation}},
  year = {2025},
  eprint = {2507.03311},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2507.03311},
}
Figure