Automatic Processing of Foreign Language Documents

G. SaltonCOLING 1969

TL;DR

This paper explores the effectiveness of automatic text processing for foreign language documents, specifically German, using the SMART retrieval system. It demonstrates that by employing a multi-lingual thesaurus, cross-language retrieval (e.g., German queries on English documents) can achieve effectiveness comparable to single-language retrieval, challenging previous assumptions about the limitations of automatic foreign language processing.

What Problem Does This Paper Address?

Traditional information retrieval systems often rely on manual indexing and human query formulation, which can be time-consuming and expensive. While automatic text processing methods have shown promise for English documents, a significant barrier to their widespread adoption has been their inability to handle foreign language materials. This paper addresses the challenge of automatically processing and retrieving information from foreign language documents, specifically German, to bridge this gap and expand the applicability of automatic retrieval systems.

Methods

The study extends the SMART (System for Mechanical Analysis and Retrieval of Text) document retrieval system to handle foreign language documents. The core of the approach involves a multi-lingual thesaurus that maps concepts from different languages to a common category. This allows for the analysis of documents and search requests in one language and their retrieval from a collection in another. The paper reviews various automatic content analysis methods within SMART, including:

  • Word stem analysis: Breaking words into their root forms.
  • Thesaurus look-up: Mapping word stems to concept numbers using a synonym dictionary.
  • Hierarchical concept arrangements: Organizing concepts to allow for generalization or specification.
  • Statistical term association: Identifying related concepts based on co-occurrence.
  • Syntactic analysis: Analyzing sentence structure to identify phrases.

The key innovation here is the development and application of a multi-lingual thesaurus that facilitates cross-language concept mapping.

Datasets

The experiments were conducted using document collections in the area of library science and documentation.

  • English Collection: 1095 document abstracts.
  • German Collection: 468 document abstracts.
  • Overlap: 50 common documents between the two collections.
  • Queries: A set of 48 search requests, originally in English, were used. German queries were created by manually translating the English queries.

Relevance assessments for English documents and queries were performed by eight American library science students. German relevance assessments were obtained from a separate German-speaking assessor.

Key Insights and Ideas

  • Cross-language retrieval is feasible and effective: The most significant finding is that automatic processing with a multi-lingual thesaurus allows for effective retrieval across languages. German queries can be matched against English documents (and vice-versa) with performance nearly as good as single-language retrieval.
  • Simplicity often trumps complexity: Consistent with earlier findings in English, simpler language analysis methods (like thesaurus-based concept matching) proved more effective than more complex syntactic analysis for retrieval.
  • Thesaurus completeness is crucial: The German thesaurus used in the study was less complete than the English one, leading to a higher number of missing words. This incompleteness negatively impacted the performance of the German document collection, highlighting the importance of comprehensive multilingual thesauri.
  • Language processing methods are transferable: The core automatic language processing techniques developed for English appear to be directly applicable to other languages like German, with minor adjustments (e.g., suffix rules).

Key Results

The study found that cross-language retrieval runs (English queries on German documents, and German queries on English documents) were not substantially inferior to single-language runs. The performance difference was on the order of 0.02 to 0.03 for a given recall level. However, runs using the German document collection (both single-language and cross-language) were generally less effective than those using the English collection. This degradation was attributed to characteristics of the German collection itself, particularly the incompleteness of the German thesaurus.

Limitations and Future Work

  • Thesaurus Completeness: The primary limitation identified is the incompleteness of the German thesaurus, which led to a significant number of words not being found. The authors suggest that a more complete thesaurus is essential for future experiments and operational use.
  • Relevance Assessments: The study notes that relevance assessments for German documents were obtained from a single German-speaking assessor, while English assessments came from a group of students. While the paper argues that this difference is unlikely to be the primary cause of performance variations, it's a potential area for further investigation.
  • Future Experiments: The authors recommend future experiments using more complete thesauri across all languages and employing identical query and document collections with consistent relevance judgments to allow for more direct cross-language comparisons. They also suggest that automatic retrieval methods used for English are likely applicable to other foreign languages.

Summary updated

BibTeX
@inproceedings{salton1969automatic,
  author = {G. Salton},
  title = {{Automatic Processing of Foreign Language Documents}},
  booktitle = {International Conference on Computational Linguistics},
  year = {1969},
}
Figure