Using Comparable Corpora to Solve Problems Difficult for Human Translators

Serge Sharoff, Bogdan Babych, Anthony HartleyACL 2006

Introduction

Translators frequently encounter obstacles when dealing with the general lexicon. While technical terminology often possesses direct equivalents, common expressions like "strong voice" or "comprehensive answer" are highly context-dependent. Traditional bilingual dictionaries frequently fail to capture these nuances, offering limited or unnatural translations. To address this, Sharoff et al. introduced ASSIST, a tool that leverages large-scale comparable corpora to suggest contextually appropriate translations for challenging general-lexicon phrases.

The ASSIST Methodology

The core innovation of ASSIST lies in its use of comparable corpora—collections of texts in two languages that cover similar genres (e.g., news) but are not direct translations. This approach avoids the "translationese" often found in parallel corpora and provides more authentic target-language expressions.

The tool operates through a four-step pipeline:

  1. Source Expansion: Identifying similarity classes for the source word (e.g., for experience, finding knowledge or feeling).
  2. Dictionary Translation: Translating the expanded set using standard bilingual resources.
  3. Target Expansion: Further expanding the target set using similarity classes in the target language.
  4. Filtering: Generating combinations and filtering them against a database of Multi-Word Expressions (MWEs).

A key technical feature is the use of negative constraints. Rather than attempting to define all correct grammatical patterns, the tool filters out impossible ones, increasing flexibility and computational efficiency.

Example of translation suggestions generated by the ASSIST tool

Performance and Evaluation

The researchers evaluated ASSIST using English and Russian corpora, including the British National Corpus (BNC) and the Russian Reference Corpus (RRC). A human evaluation involving eight professional translators compared ASSIST's suggestions against authoritative dictionaries.

The study identified two distinct categories of translation problems:

  • Agreement Group: Scenarios where dictionaries agree and provide high-quality solutions.
  • Disagreement Group: "Recalcitrant" cases where dictionaries offer little help and human translators vary significantly in their assessments.
Comparison of dictionary and ASSIST performance in agreement and disagreement scenarios

Results indicated that ASSIST achieved parity with top-tier dictionaries overall, with an average score of 4.2/5. Crucially, in the "disagreement group," ASSIST outperformed dictionaries, receiving significantly higher ratings (3.97 vs. 3.77). Furthermore, ASSIST's suggestions demonstrated greater consistency across different evaluators, as evidenced by lower standard deviation in scores compared to dictionary results in difficult cases.

Limitations and Future Directions

Despite its success, the current model faces challenges such as syntactic mirroring, where it tends to suggest target phrases with the same grammatical structure as the source. It also lacks full context awareness, as it generates suggestions for phrases in isolation. Future work aims to integrate semantic tagging to better identify the "semantic signature" of specific contexts, further refining the filtering process.

Conclusion

ASSIST demonstrates that comparable corpora are a powerful resource for solving translation problems that traditional dictionaries cannot handle. By focusing on authentic language use rather than rigid definitions, the tool provides translators with robust, contextually appropriate options for the most difficult parts of the general lexicon.

Reference: Sharoff, S., Babych, B., & Hartley, A. (2006). Using comparable corpora to solve problems difficult for human translators. Proceedings of the COLING/ACL 2006.

Summary updated

BibTeX
@inproceedings{sharoff2006using,
  author = {Serge Sharoff and Bogdan Babych and Anthony Hartley},
  title = {{Using Comparable Corpora to Solve Problems Difficult for Human Translators}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2006},
}
Figure