An Unsupervised System for Identifying English Inclusions in German Text

Beatrice AlexACL 2005

Abstract

Identifying English inclusions, or anglicisms, within German text is a significant challenge for Natural Language Processing (NLP) tasks such as Text-to-Speech and Machine Translation. This paper introduces an unsupervised system that combines lexicon lookups with real-time statistics from the web to classify nouns by language. Unlike supervised models that require extensive hand-annotated data and often fail when applied to new domains, this system offers high portability and robustness. Evaluation across three distinct domains—Internet, Space, and the European Union—shows that the unsupervised approach achieves competitive F-scores and significantly outperforms standard language identification tools like TextCat.

Problem Statement

As global interconnectedness grows, languages frequently borrow terms from one another. In German, English inclusions are rampant in sectors like technology, advertising, and science (e.g., "Security-Tool," "Hacker," "E-Business"). Standard NLP tools often struggle with these inclusions; for instance, a Text-to-Speech (TTS) system must identify if a word is English to apply the correct pronunciation, and Machine Translation (MT) systems must decide whether to translate or pass through foreign terms.

The challenge lies in the dynamic nature of language: new words enter the lexicon constantly, making static lists obsolete. While supervised machine learning models can address this, they require expensive human-annotated data and are often "brittle," failing when moved to a new topic or domain.

Methodology

The authors developed a UNIX-based pipeline that classifies nouns as either German or English using a three-tiered approach:

  1. Pre-processing: The system cleans HTML, tokenizes text, and applies Part-of-Speech (POS) tagging to isolate nouns and named entities.
  2. Lexicon Lookup: Tokens are checked against the CELEX database. Words found only in the English lexicon are tagged as English, while homographs (words appearing in both, like Rat) are tagged as German by default.
  3. The "Web-as-a-Corpus" Strategy: For words missing from the lexicon, the system queries Google to compare hits on German-language versus English-language pages. Because English web pages vastly outnumber German ones, a normalization technique is applied to ensure a fair comparison.
Normalization of search engine hits for German and English terms.
  1. Post-processing: Rule-based grammars identify specific cases, such as English prefixes in hyphenated compounds (e.g., the "E" in "E-Mail").

Datasets and Evaluation

The researchers created a manually annotated "gold standard" corpus from Frankfurter Allgemeine Zeitung articles (2001–2004) across three domains: Internet & Telecoms, Space Travel, and the European Union. The total corpus size was approximately 48,000 tokens. Notably, English inclusions were frequent in the tech domain (nearly 1,000 tokens) but extremely rare in the EU domain (only 49 tokens).

Key Results

The unsupervised system demonstrated strong performance and significant portability. In the Internet domain, the "Lookup + post" method achieved an F-score of 77.2%, while the Space domain reached 73.7%.

Performance comparison of unsupervised methods and baselines across three domains.

The "Lazy Learner" Problem

A critical insight from the study is the brittleness of supervised models. While a supervised tagger reached F-scores over 90% when trained and tested on the same domain, its performance plummeted to as low as 22.2% in cross-domain tests (e.g., training on Internet data but testing on Space data). This suggests that supervised models often "memorize" domain-specific lexicons rather than learning general features of the language. In contrast, the unsupervised system's reliance on the massive, updating resource of the Web makes it far more robust for real-world applications.

Limitations and Future Work

Despite its success, the system faces challenges with morphological complexity, such as "mixed" words where English roots take German inflections (e.g., Receivern). Future iterations may require a sophisticated morphological analyzer to strip German endings. Additionally, interlingual homographs remain difficult to distinguish without deeper semantic context analysis. Finally, the system is currently constrained by the limitations of search engine APIs, such as a lack of case or POS sensitivity.

Summary updated

BibTeX
@inproceedings{alex2005unsupervised,
  author = {Beatrice Alex},
  title = {{An Unsupervised System for Identifying English Inclusions in German Text}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2005},
}
Figure