A Practical Chinese-English ON Translation Method Based on ON‘s Distribution Characteristics on the Web

Feiliang RenCOLING 2012

TL;DR

This paper tackles the challenge of translating Chinese organization names (ONs) into English by leveraging their distribution characteristics on the web. The authors propose a novel web mining method that effectively identifies correct translations, improving both inclusion rates and translation quality metrics like BLEU score.

What Problem Does This Paper Address?

Translating named entities (NEs), particularly organization names (ONs), is crucial for many Natural Language Processing (NLP) tasks like cross-language information retrieval and machine translation. While the web offers a vast resource for finding correct translations, a key challenge is effectively searching and identifying these translations within the noisy web data. Existing methods often rely on assumptions about the presence of both the source and target ONs in mixed-language web pages. This paper questions the universal validity of these assumptions and aims to validate them experimentally. Furthermore, it proposes a more robust web mining approach that accounts for the specific distribution patterns of Chinese ONs online.

Methods

The core of the paper's contribution lies in its proposed web mining method for Chinese-English ON translation. The method is motivated by experimental validation of two key assumptions:

  1. Assumption 1: The correct translation of a Chinese ON exists on the web.
  2. Assumption 2: Both the original Chinese ON and its English translation appear together in some mixed-language web pages.

The authors first experimentally validate these assumptions by constructing various queries (monolingual Chinese, monolingual English, and bilingual) and analyzing their inclusion rates in search results. Based on the observed distribution characteristics of Chinese ONs on the web, they develop a web mining algorithm. This algorithm involves:

  • Segmentation: Breaking down the Chinese ON into smaller, meaningful chunks (e.g., regionally restrictive, keyword, and middle specification chunks).
  • Candidate Generation: Generating translation candidates for the original ON and its segmented chunks using a pre-trained translation system (NEUTrans).
  • Query Construction: Creating a diverse set of queries by combining the original ON and its translated chunks in various ways.
  • Query Revision: Employing specific rules to refine queries based on their presence and similarity to content found in returned web pages, addressing issues like reordering and out-of-vocabulary terms.
  • Result Selection: Recommending a translation based on the revised queries, prioritizing those that are well-supported by web evidence.

Datasets

  • Test Data for Assumption Validation: The paper uses bilingual ON translation pairs extracted from LDC2005T34. These pairs are further categorized into 20 groups based on keyword types for detailed analysis.
  • Test Data for Method Evaluation: For the second experiment, 500 Chinese ONs were randomly selected from those whose correct translations do not exist on the web. These ONs were manually chunked.
  • Translation Candidate Generation: The NEUTrans system is used to generate translation candidates. Its training corpus consists of approximately 370K bilingual sentences extracted from LDC2005T10, LDC2003E07, LDC2003E14, and LDC2005T06.

Key Insights and Ideas

  • Assumption Validation is Crucial: The paper empirically demonstrates that while correct translations for most Chinese ONs do exist on the web, simply using bilingual queries is insufficient to find them effectively. The inclusion rates for bilingual queries (Q3) are significantly lower than for monolingual English queries (Q2).
  • Distribution Matters: The distribution characteristics of Chinese ONs on the web vary significantly. ONs that appear frequently online (e.g., multinational companies, universities) are easier to translate than those that are rare or have complex structures.
  • Chunking Improves Discoverability: For ONs that are rarely found on the web, segmenting them into smaller chunks (e.g., "Regionally Restrictive Chunk," "Keyword Chunk," "Middle Specification Chunk") dramatically improves the inclusion rate of these smaller units. This suggests that breaking down complex ONs into their constituent parts makes them more discoverable.
  • Web Mining Needs Sophistication: Simple web searches are not enough. The proposed method's query revision rules are essential for handling the complexities of web data, including reordering and out-of-vocabulary terms, to extract reliable translation candidates.

Key Results

  • Assumption Validation: The inclusion rate for monolingual English queries (Q2) is consistently much higher than for bilingual queries (Q3), indicating that finding the English part of the translation directly is often more effective than searching for both parts together. The Q3' metric (English part contained in returned pages) shows that even with bilingual queries, the inclusion rate is still lower than Q2.
  • Chunking Effectiveness: Table 2 shows that for ONs whose English translations are not found on the web, chunking leads to 100% inclusion rates for the individual chunks (Q1 and Q2). The inclusion rate for chunk translation pairs (Q3 and Q3') also shows improvement compared to full ON translation.
  • Method Performance: The proposed web mining method (labeled "Our" in Table 3) significantly improves translation performance. For a dataset of 500 Chinese ONs whose correct translations don't exist on the web, the method achieved a BLEU score of 0.2326 and a Top1 Accuracy of 19.4%, compared to a baseline (NEUTrans) of 0.1811 and 11.6% respectively. This demonstrates a substantial improvement in translation quality.

Limitations and Future Work

  • Manual Chunking: The evaluation experiment for chunking involved manual chunking of ONs. The paper suggests that automating this chunking process would be a valuable direction for future work.
  • OOV Translation Problem: While the query revision rules attempt to address out-of-vocabulary (OOV) terms, this remains a complex challenge in ON translation.
  • Generalizability: The study focuses on Chinese-English ON translation. Further research could explore the applicability of these methods to other language pairs and NE types.
  • Web Resource Dependence: The effectiveness of the method relies heavily on the availability and quality of web resources. The performance might vary across different languages or domains with less web presence.

Summary updated

BibTeX
@inproceedings{ren2012practical,
  author = {Feiliang Ren},
  title = {{A Practical Chinese-English ON Translation Method Based on ON‘s Distribution Characteristics on the Web}},
  booktitle = {International Conference on Computational Linguistics},
  year = {2012},
}
Figure