An Automatic Post Editing With Efficient and Simple Data Generation Method

Hyeonseok Moon, Chanjun Park, Jaehyung Seo, Sugyeong Eo, Heuiseok LimIEEE Access 2022 · DOI · OpenReview

Introduction

Automatic Post-Editing (APE) serves as a crucial second-pass mechanism in machine translation (MT) pipelines, aimed at correcting systematic errors produced by primary MT engines. Despite its potential, the development of robust APE models is severely hindered by the "data desert" phenomenon. Traditional training requires triplets consisting of a source sentence ($src$), a machine-translated output ($mt$), and a human-corrected post-edited version ($pe$). The high cost and slow pace of manual post-editing make such datasets scarce for most language pairs, limiting APE research primarily to high-resource languages like English and German. This study addresses this bottleneck by proposing a method to generate high-quality synthetic training data from standard parallel corpora.

Methodology

The core of the proposed approach is a "break-it-to-fix-it" strategy. By taking a high-quality parallel corpus (containing $src$ and gold-standard $pe$), the researchers intentionally degrade the $pe$ sentences to create synthetic $mt$ counterparts.

Synthetic Data Generation

Three distinct noising techniques were evaluated to simulate machine translation errors:

  1. Random Noise: Involves random insertion, deletion, or swapping of words within the sentence.
  2. POS-Based Noise: Utilizes Part-of-Speech (POS) tagging to replace words with others of the same category (e.g., replacing a verb with another random verb).
  3. Semantic Level Noise: Employs WordNet to replace words with their synonyms.
Overview of the synthetic data generation process using POS tagging and WordNet to create various types of noise.

Efficient Training with Bottleneck Adapter Layers

To ensure computational efficiency, the researchers utilized Bottleneck Adapter Layers (BAL). Instead of fine-tuning the entire pre-trained Transformer model (98 million parameters), they inserted small, trainable layers and froze the original weights. This allowed the model to adapt to the APE task by training only 1.6 million parameters (approximately 1.6% of the total), significantly reducing resource requirements without sacrificing performance.

Experimental Setup

The models were trained and validated using 120,000 sentence pairs from the AIhub Korean-English parallel corpus, a high-quality dataset verified by human experts. To evaluate real-world applicability, the APE models were tested against translations produced by three major commercial engines: Google Translate, Microsoft Translator, and Amazon Translate.

Results and Discussion

The experimental results demonstrated a significant performance leap, particularly when using POS-based noise. On the Amazon Translate test set, the baseline (no APE) recorded a Translation Edit Rate (TER) of 60.42 and a BLEU score of 23.32. The POS-noised APE model improved these metrics to 45.80 TER and 43.42 BLEU—a 14.6-point reduction in error rate and a 20-point increase in translation quality.

The Importance of Structure

A key insight was that POS-based noise outperformed other methods because it maintains the grammatical structure of the sentence. By replacing a word with another of the same POS, the model is forced to learn context and semantic nuances rather than simply fixing broken syntax.

Conversely, Semantic Level Noise performed the worst. The authors attribute this to the context-blind nature of WordNet synonyms; replacing a word like "help" with "assist" might be semantically accurate in isolation but can be stylistically inappropriate or unnatural in specific contexts, leading to poor training signals.

Mutual Supplementation

The study also found a "supplementation effect" when combining synthetic noised data with traditional back-translation data. This hybrid approach yielded better results than using either method in isolation, suggesting that synthetic noise and MT-generated errors provide complementary learning signals.

Conclusion

This research provides a scalable solution to the APE data scarcity problem. By demonstrating that POS-aware synthetic noise can effectively simulate MT errors, the authors enable the training of high-performance APE models for low-resource languages without the need for expensive human intervention. Future work will focus on developing more context-aware noising techniques and scaling these methods for industrial-grade, data-centric translation services.

Summary updated

Abstract

Automatic post-editing (APE) research considers methods for correcting translation results inferred by machine translation systems. The training of APE models, generally require triplets including a source sentence ( <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$src$ </tex-math></inline-formula> ), machine translation sentence ( <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$mt$ </tex-math></inline-formula> ), and post-edited sentence ( <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$pe$ </tex-math></inline-formula> ). As considerable expert-level human labor is required in creating <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$pe$ </tex-math></inline-formula> , APE researches have encountered difficulty in constructing suitable dataset for most of language pairs. This has led to the absence of APE data for most of language pairs, such as Korean-English, and imposed limitation to the sustainable researches of APE. Motivated by this problem, we propose a method that can generate APE triplets using only a parallel corpus without human labor. Our proposal comprises three noise generation techniques, including random, part of speech tagging (POS) based, and semantic level noises, and the effectiveness of these methods are verified by the results of quantitative and qualitative experiments on Korean-English APE tasks. As a result of our experiments, we find that POS based noise encourages the best APE performance. The proposed method is influential in that it can obviate expert human labor which was generally required in APE data construction, and enable the sustainable APE researches for the most language pairs where human-edited APE triplets are unavailable.
BibTeX
@article{moon2022automatic,
  author = {Hyeonseok Moon and Chanjun Park and Jaehyung Seo and Sugyeong Eo and Heuiseok Lim},
  title = {{An Automatic Post Editing With Efficient and Simple Data Generation Method}},
  journal = {IEEE Access},
  year = {2022},
  doi = {10.1109/access.2022.3152001},
}
Figure