Automatic Bilingual Markup Transfer

Joern Wuebker, Thomas Zenkel, John DeNeroACL 2021 · OpenReview

Abstract: We describe the task of bilingual markup transfer, which involves placing markup tags from a source sentence into a fixed target translation. This task arises in practice when a human translator generates the target translation without markup, and then the system infers the placement of markup tags. This task contrasts from previous work in which markup transfer is performed jointly with machine translation. We propose two novel metrics and evaluate several approaches based on unsupervised word alignments as well as a supervised neural sequence-to-sequence model. Our best approach achieves an average accuracy of 94.7% across six language pairs, indicating its potential usefulness for real-world localization tasks.

Introduction

In professional localization, the translation process often occurs in two distinct stages. First, human translators produce high-quality raw text translations. Second, formatting tags—such as XML tags for bolding, italics, or hyperlinks—must be manually re-inserted into the target text. This manual re-insertion is both tedious and prone to error.

This paper addresses the "assistance" use case: automatically transferring markup from a source sentence into a fixed reference translation. This is a significant challenge because human translations often deviate from the literal structure of the source, and even a single incorrect word alignment can cause a tag to be misplaced across the sentence.

Methodology

The researchers explored three primary strategies for solving the markup transfer problem:

1. Unsupervised Alignment and Projection

Traditional methods use word aligners (like FastAlign) to map source words to target words. The standard "Min-Max" approach simply wraps a tag around the first and last aligned words in the target. However, this is fragile; one bad alignment link can "drag" a tag across the entire sentence.

To counter this, the authors proposed the Inside-Outside span projection algorithm. This method scores all possible target spans based on the density of alignment links falling inside versus outside the span, making it significantly more resilient to spurious links.

Comparison of Min-Max vs. Inside-Outside projection methods. The Inside-Outside approach correctly identifies the intended span despite complex alignments.

2. Supervised Sequence-to-Sequence

A Transformer-based model was trained to take a tagged source and a tag-less target as input, generating the target with tags inserted. To ensure the model preserved the human's original translation, the authors employed constrained search, forcing the output to match the provided target tokens exactly while only allowing the model to decide tag placement.

3. The Hybrid Approach

While the supervised model is highly accurate, it can occasionally produce inconsistent outputs. The best-performing system is a hybrid that utilizes the supervised model's output when it is consistent and falls back to the unsupervised NeuralAlign method otherwise.

Experimental Results

The models were evaluated on a dataset of approximately 100,000 segments from enterprise software documentation across six language pairs (English to German, Finnish, French, Japanese, Dutch, and Chinese).

Performance of various markup transfer methods across six language pairs. The top number in each cell represents Tag Accuracy, while the bottom number represents Average Character Distance.

Key Findings

  • Superior Accuracy: The hybrid Seq2Seq + NeuralAlign model achieved a 94.7% average accuracy, a massive improvement over the 72.3% baseline.
  • Reduced Error Severity: Beyond just accuracy, the "severity" of errors dropped. The average character distance for misplaced tags fell from 5.3 in the baseline to just 1.3 in the final model.
  • Neural Alignment Impact: Switching from statistical aligners (FastAlign) to neural-based aligners (NeuralAlign) provided a significant boost in unsupervised performance, jumping from ~72% to ~83%.
  • The "Perfect Match" Heuristic: Prioritizing exact matches for phrases that appear identically in both languages (e.g., product names) provided a consistent 1.6% accuracy gain.

Discussion and Limitations

Japanese proved to be the most challenging language for the system. Specific hurdles included UI elements that require brackets in Japanese but not in English, and grammatical particles that are often "glued" to content words during tokenization, causing tags to occasionally include or exclude necessary characters.

Future research may focus on more sophisticated subword tokenization to prevent this "bleeding" between content and grammar. Additionally, exploring ways to achieve high accuracy without the need for large, specialized datasets of tagged bitext remains a priority for making these systems more accessible.

Summary updated

Abstract

We describe the task of bilingual markup transfer, which involves placing markup tags from a source sentence into a fixed target translation. This task arises in practice when a human translator generates the target translation without markup, and then the system infers the placement of markup tags. This task contrasts from previous work in which markup transfer is performed jointly with machine translation. We propose two novel metrics and evaluate several approaches based on unsupervised word alignments as well as a supervised neural sequence-to-sequence model. Our best approach achieves an average accuracy of 94.7% across six language pairs, indicating its potential usefulness for real-world localization tasks.
BibTeX
@inproceedings{wuebker2021automatic,
  author = {Joern Wuebker and Thomas Zenkel and John DeNero},
  title = {{Automatic Bilingual Markup Transfer}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2021},
  url = {https://openreview.net/forum?id=RaDfoiOmuZ},
}
Figure