TL;DR
This paper introduces ADOMIT, a novel algorithm for automatically detecting accidental omissions in translations. ADOMIT achieves this by analyzing the geometric properties of "bitext maps" (visual representations of text alignment) without relying on linguistic information, making it robust to various error types. This approach offers a valuable quality control tool for translators, even with imperfect underlying text alignment technology.
What Problem Does This Paper Address?
Translating large volumes of text is increasingly driven by cost pressures, leading to a reduction in manual proofreading. This creates a need for automated methods to detect errors, particularly omissions, where a translator might accidentally skip a sentence or paragraph. While manual proofreading can catch these, it's becoming economically unfeasible. Existing methods for aligning texts (creating "bitext maps") are imperfect, and detecting omissions reliably requires a method that can handle the noise and inaccuracies inherent in these maps. The paper aims to fill this gap by providing an algorithm that can identify these omissions using only the geometric information from bitext maps.
Methods
The core of ADOMIT lies in its geometric analysis of bitext maps. A bitext map visualizes the correspondence between segments of an original text and its translation, typically plotted in a 2D space where axes represent character positions in each text.
- Bitext Map Construction: The paper assumes the existence of a bitext map, which is a sequence of corresponding character positions. These points, when interpolated, form a function representing the alignment.
- Geometric Properties: The key insight is that in a perfect translation (mutual translations), the bitext map function is injective (one-to-one) and exhibits a low variance in segment slopes. This is because the lengths of mutual translations are highly correlated.
- Omission Detection (Basic Method): An omission in the original text corresponds to a region in the bitext map with an unusually low slope (nearly horizontal if the original text is on the y-axis and translation on the x-axis). The "Basic Method" flags segments with a slope angle below a predefined threshold
t. - ADOMIT (Robust Variation): ADOMIT addresses the problem of "noisy" bitext maps, which are common with automatic alignment tools. Noise can cause fragmentation of genuine omissions into smaller segments or create spurious "omitted" segments. ADOMIT reconstructs "maximal omitted segments" by efficiently searching for and concatenating "minimal omitted segments" that likely originated from a single, larger omission, using geometric theorems to speed up this search. This makes it more robust to alignment errors.
Datasets
The paper primarily uses hand-constructed bitext maps derived from the Canadian Hansards (parliamentary debate transcripts available in English and French). These are referred to as the "easy" and "hard" bitexts and serve as a "gold standard" for evaluating bitext mapping and alignment algorithms. The authors also created simulated omissions within these bitexts for quantitative evaluation.
Key Insights and Ideas
- Geometry over Linguistics: The most striking idea is that omission detection can be achieved purely through geometric analysis of bitext maps, completely ignoring linguistic content. This makes the algorithm language-independent and capable of detecting non-linguistic errors (e.g., from word processing mishaps).
- Bitext Maps as a Diagnostic Tool: The paper demonstrates that even "gold standard" hand-aligned bitexts contain errors, both in alignment and in the original translations. ADOMIT's ability to find these errors highlights the inherent challenges in creating perfect bitexts.
- Noise Resilience: ADOMIT's core contribution is its robustness to noisy bitext maps. By intelligently reconstructing maximal omitted segments from fragmented minimal ones, it overcomes a major limitation of simpler geometric approaches.
- Intended vs. Unintended Omissions: The paper acknowledges that not all omissions are errors. It proposes a practical heuristic: accidental omissions tend to be much longer than intended paraphrases, allowing for a simple sorting mechanism to prioritize likely errors.
Key Results
- ADOMIT successfully identified numerous errors in hand-aligned "gold standard" bitexts, including omissions, alignment errors, and accidental repetitions.
- In simulated evaluations, ADOMIT significantly outperformed the "Basic Method" for omission detection, achieving up to 48 percentage points higher recall for simulated translators with varying patience levels.
- ADOMIT demonstrated robustness, as evidenced by shorter confidence intervals in its recall scores compared to the Basic Method.
- The algorithm can help translators catch a high percentage of paragraph-sized omissions (over 90%) and a substantial portion of sentence-sized omissions (over 50%), even with imperfect bitext maps.
Limitations and Future Work
- Bitext Map Quality: ADOMIT's performance is fundamentally limited by the quality of the input bitext map. While it's robust to noise, extremely poor maps will still hinder its effectiveness.
- Language Pair Testing: The current evaluation was performed on language pairs for which the SIMR bitext mapping algorithm reliably produces good maps. Future work should test ADOMIT on other language pairs to assess its generalizability.
- Integration into Translator Workflow: While the paper suggests ADOMIT can be easily implemented and integrated, further research could explore optimal user interfaces and workflows for translators.
- Further Refinement of Noise Handling: While ADOMIT is robust, there's always room for improvement in handling various types of noise in bitext maps.
Summary updated