InDeep $\times$ NMT: Empowering Human Translators via Interpretable Neural Machine Translation

Gabriele Sarti, Arianna BisazzaEAMT 2022

Introduction

Neural Machine Translation (NMT) has reached a level of performance where its outputs are often indistinguishable from human translation. However, these systems operate as "black boxes"—complex, multi-layered architectures whose decision-making processes are opaque to the end user. For professional translators, this lack of transparency is more than a theoretical concern; it creates a "post-editing" environment where they must correct errors without understanding why they occurred. The InDeep project (2021–2026) seeks to bridge this gap by developing interpretable NMT systems that empower human experts to move from passive proofreading to active, informed collaboration with AI.

The Challenge of Opaque Translation

While NMT models are highly capable, they are prone to specific failure modes, such as "hallucinations" (generating content not present in the source) or critical omissions. Currently, most interpretability research is largely confined to the theoretical domain, focusing on model architecture rather than user utility. This leaves professional translators in a difficult position: they are responsible for the final quality of a text but lack the tools to interrogate the machine's choices. This opacity hinders accountability and increases the cognitive load of the translation process.

Methodology: A Two-Pronged Approach

The InDeep project proposes a framework that combines technical innovation with human-centric evaluation to bridge the gap between AI theory and translation practice:

1. Methodological Development

The project focuses on adapting two primary forms of interpretability:

  • Feature Attribution: Identifying which specific segments of the source text most influenced a particular word or phrase in the translation.
  • Instance Attribution: Pointing to specific examples within the training dataset that drove the model’s current prediction.

A critical component of this development is ensuring faithfulness. For an explanation to be useful, it must accurately reflect the model's internal logic. If an attribution tool provides a "plausible" but incorrect explanation, it risks misleading the translator and undermining the entire process.

2. User-Centric Evaluation

To move beyond theoretical metrics, InDeep employs behavioral tracking to monitor how professional translators interact with interpretable outputs. By using keylogging, eye-tracking, and mouse-tracking, researchers can measure cognitive load and efficiency in real-world scenarios, ensuring that the tools actually improve the workflow.

Evaluation and Data

As a project roadmap and description, InDeep utilizes a combination of automated and human-centric data sources for evaluation:

  • Faithfulness Metrics: Quantitative measures to ensure interpretability tools are causally linked to model output.
  • Field Study Data: Collected in partnership with professional agencies (specifically GlobalTextware) to capture the nuances of industry workflows.
  • Behavioral Logs: Detailed records of keystrokes and eye movements to quantify the impact of interpretability on translator efficiency and cognitive load.

Key Insights for Human-AI Collaboration

The project is guided by several core philosophies regarding the future of the translation industry:

  • Active Agency: The goal is to shift the human role from a passive recipient of AI output to an active driver of the translation process. By providing intelligible feedback, the system empowers the translator to steer the AI.
  • Practical Grounding: Interpretability is only valuable if it assists a user in performing a task. Knowing how a model works is secondary to knowing how that knowledge helps the translator.
  • Clarity over Complexity: To avoid information overload, the project emphasizes presenting interpretability data without visual clutter, recognizing that too much information can be as detrimental as none at all.

Preliminary Results and Future Directions

While final quantitative results are pending as the project progresses, preliminary inquiries with professionals have identified two high-value directions for the technology:

  • Sentence-Level Quality Estimation: Allowing translators to quickly triage documents and focus on sections requiring the most attention.
  • Adaptive Constraints: Enabling translators to enforce specific terminology or stylistic requirements that the model must respect.

The primary challenge remains the scalability and generalizability of these methods, as many current interpretability techniques are computationally intensive or model-specific. Future work will focus on creating "self-contained" methods for standard workflows and conducting extensive field studies to quantify gains in both efficiency and translator "enjoyability."

Conclusion

By opening the "black box" of NMT, the InDeep project aims to create a more symbiotic relationship between human translators and AI. Through faithful attribution and user-focused design, the project seeks to restore agency to human experts, ensuring that the future of translation is one of empowered collaboration rather than automated replacement.

Summary updated

BibTeX
@inproceedings{sarti2022indeep,
  author = {Gabriele Sarti and Arianna Bisazza},
  title = {{InDeep $\textbackslash\{\}times$ NMT: Empowering Human Translators via Interpretable Neural Machine Translation}},
  booktitle = {EAMT},
  year = {2022},
}
Figure