Semi-Supervised Lexicon Mining from Parenthetical Expressions in Monolingual Web Pages

Xianchao Wu, Naoaki Okazaki, Jun'ichi TsujiiNAACL 2009 · OpenReview

Xianchao Wu, Naoaki Okazaki, Jun’ichi Tsujii
Proceedings of Human Language Technologies: The 2009 Annual Conference of the North American Chapter of the Association for Computational Linguistics. 2009.

Abstract

This paper introduces a semi-supervised framework to automatically build Chinese-English lexicons by mining "parenthetical expressions" (e.g., "Chinese Term (English Translation)") from massive web crawls. By combining frequency-based term recognition with self-training transliteration and translation models, the authors successfully extracted over a million bilingual entries, providing a scalable way to keep up with ever-evolving neologisms.

Introduction: The Challenge of Evolving Lexicons

Bilingual lexicons are foundational to Machine Translation (MT) and Cross-Lingual Information Retrieval. However, human language evolves faster than manual lexicography can keep pace. New technical terms, brand names, and neologisms appear daily on the web, often lacking entries in traditional dictionaries.

Traditional mining methods typically rely on scarce parallel corpora or monolingual corpora that lack sufficient context for accurate mapping. This research exploits a common "social convention" on the web: writers in Asian languages often annotate new or difficult terms by placing the English translation inside parentheses immediately following the term. The primary challenges involve automatically extracting these pairs, solving the "boundary problem" (determining where the Chinese term starts), and classifying the linguistic relationship (abbreviation, transliteration, or semantic translation).

Examples of different parenthetical expression types found in Chinese web pages, illustrating the boundary problem marked by ||.

Methodology

The authors propose a multi-stage pipeline to process parenthetical expressions extracted from massive web crawls. The framework categorizes expressions into three distinct linguistic types to apply specialized mining strategies.

The multi-stage pipeline for lexicon mining from parenthetical expressions.

1. Bilingual Abbreviation Extraction

To identify if a Chinese string is the "long-form" of an English acronym found in parentheses, the system uses a frequency-based LH (Long-form Likelihood) score. This metric evaluates how often a candidate string and an acronym co-occur compared to their sub-sequences, helping to distinguish valid pairs from noise.

2. Self-Training for Transliteration

For proper names (e.g., "Shipton-Tilman"), the authors developed a Hybrid Model that maps English syllables to Chinese Pinyins. Utilizing the Moses SMT toolkit, they treated transliteration as a translation problem at the character/syllable level. This hybrid approach strikes a balance between phonetic accuracy and the complex one-to-many mapping inherent in Chinese characters.

3. Self-Training for Translation

Technical terms are handled via a Cascaded Translation Model. Using the Morfessor tool, the system segments words into morphemes (prefixes, stems, suffixes). This allows the model to learn semantic correspondences at a granular level, enabling it to translate previously unseen words by recognizing their components (e.g., identifying the Chinese equivalent for the medical suffix "-itis").

4. The Self-Training Loop

Both transliteration and translation models utilize a self-training loop. Starting with a small "seed" lexicon of labeled data, the models predict translations for unlabeled web data. The highest-confidence pairs are then iteratively added back into the training set to refine the models.

Datasets and Evaluation

The system was trained and evaluated using several large-scale resources:

  • SogouT Internet Corpus: ~13 billion Chinese web pages (252 GB).
  • Peking University Chinese Paper Corpus: 55 GB of academic text.
  • LDC Named Entity Lists & Wanfang Technical Dictionary: Used for initial seed training.
  • Wikipedia: Served as a "gold standard" evaluation set by extracting linked Chinese and English article titles.

Key Results and Insights

  • Scale of Extraction: The framework successfully extracted 1,060,936 translations, 107,856 abbreviations, and 50,313 transliterations.
  • Performance: Manual evaluation of abbreviations showed an accuracy of 60.5% after linguistic filtering. The hybrid transliteration model achieved a pinyin-based Word Error Rate (WER) of 39.13%, competitive with contemporary supervised systems.
  • Morpheme Advantage: Breaking terms into morphemes proved essential for handling the long-tail of technical neologisms that do not appear in standard dictionaries.

Limitations and Future Work

Despite its success, the "boundary problem" remains a significant hurdle; functional words often get incorrectly included in the extracted Chinese terms. Furthermore, the system is highly dependent on the quality of the initial seed lexicon. Future research may focus on implementing confidence-based weighting functions to improve the stability of the self-training iterations and better filter noise from the extraction process.

Summary updated

BibTeX
@inproceedings{wu2009semi,
  author = {Xianchao Wu and Naoaki Okazaki and Jun'ichi Tsujii},
  title = {{Semi-Supervised Lexicon Mining from Parenthetical Expressions in Monolingual Web Pages}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2009},
  url = {https://openreview.net/forum?id=r1lyu7bd-S},
}
Figure