A Layered Approach to NLP-Based Information Retrieval

Sharon FlankCOLING 1998

TL;DR

This paper introduces "Intermezzo," a layered approach to information retrieval that integrates multiple search engines and databases through a natural language processing (NLP) layer. This NLP layer enhances queries by performing morphological analysis, noun phrase syntax identification, and semantic expansion using WordNet, enabling more sophisticated and precise retrieval from diverse data sources.

What Problem Does This Paper Address?

Traditional information retrieval systems often struggle with the complexity and diversity of user queries and data sources. Users may not know the exact keywords to use, and data can be spread across multiple databases with different indexing schemes. Furthermore, simple keyword matching can lead to irrelevant results or miss relevant information due to variations in language, synonyms, or related concepts. This paper addresses the challenge of building a flexible and intelligent retrieval system that can understand natural language queries and effectively search across multiple, heterogeneous information sources.

Methods

The core of the approach is a "layered" architecture. The top layer is an abstraction layer that allows simultaneous querying of multiple databases, translating user queries into formats compatible with various search engines (e.g., Fulcrum, RetrievalWare). The crucial component is the NLP layer, which processes user queries before they are sent to the search engines. This NLP layer performs several key tasks:

  • Tokenization and Part-of-Speech Tagging: Queries are broken down into individual words and phrases, and their grammatical roles are identified.
  • Morphological Analysis: Word roots are extracted by stripping affixes, simplifying queries and enabling matching of variations (e.g., "disaster" and "disasters"). Derivational morphology is not yet implemented.
  • Semantic Expansion: This is a key innovation. Using WordNet, the system expands queries to include synonyms, hypernyms, and other related concepts. This expansion is constrained by part-of-speech and context to maintain precision. For example, a query for "natural disasters" might be expanded to include "hurricane," "earthquake," or "tornado," and related locations.
  • Name and Location Recognition (Gazetteer Integration): The system identifies and expands proper names and locations, leveraging a large gazetteer and specialized algorithms to disambiguate and prioritize results (e.g., distinguishing between New York state and New York City).
  • Syntactic and Other Patterns: The system identifies noun phrases and other syntactic structures to improve query understanding and ranking.

The system also employs a weighting mechanism for semantic expansion, where more closely related terms receive higher weights, influencing the ranking of search results.

Datasets

  • WordNet 1.4: Used extensively for semantic expansion, providing a lexical database of English words. The paper notes that WordNet senses were reordered by frequency based on a corpus of 150,000 photo captions.
  • Gazetteer: A large dataset, likely derived from U.S. Government sources (mentioned in the context of Message Understanding Conferences - MUC), used for location recognition.
  • Photo Captions: A corpus of approximately 150,000 to 500,000 photo captions was used for tuning the NLP component and for evaluating the semantic net's sense ordering.
  • Test Set: A manually created test set of 1200 images was used for initial evaluation, along with live system testing on a corpus of 500,000 images.

Key Insights and Ideas

  • Layered NLP for IR: The paper demonstrates the effectiveness of a multi-layered NLP approach, where each layer (morphological, semantic, syntactic) contributes to a richer query representation, rather than relying on a single NLP technique.
  • Constrained Semantic Expansion is Crucial: Simply expanding queries with all related terms from WordNet would lead to a drastic loss of precision. The paper highlights the importance of constraining expansion by part-of-speech and context.
  • Gazetteer Integration for Location Queries: Effectively handling location-based queries requires a robust gazetteer and intelligent disambiguation algorithms to manage the vastness and ambiguity of geographical names.
  • Weighting for Relevance: Assigning weights to semantic expansion terms and syntactic structures is vital for ranking results and ensuring that the most relevant items appear at the top.
  • Practical Trade-offs: The paper acknowledges that full NLP processing can be too slow for real-time retrieval. Therefore, semantic expansion is often performed offline on the caption database, and the system balances the depth of NLP processing with retrieval speed.

Key Results

  • Improved Retrieval: The system, named Intermezzo, allows for simultaneous querying of multiple databases.
  • NLP Component Performance: Retrieval speed for the NLP component averages under a second.
  • Evaluation Metrics: In one retrieval test, the combination of noun phrase syntax and name recognition improved recall by 18% at a fixed precision point.
  • Precision and Recall: For a test set, precision was measured at 89.6% and recall at 92%.
  • Sense Tagging Impact: Preliminary experiments showed that a sense-tagged corpus resulted in higher precision (99%) compared to an untagged corpus (89%), although the paper notes these figures are "artificially inflated."

Limitations and Future Work

  • Derivational Morphology: Not currently implemented, which could limit the system's ability to handle certain word transformations.
  • Preposition Handling: The loss of prepositions in some cases presents a problem, particularly for queries involving spatial relationships, and requires further investigation.
  • Speed and Space Optimizations: Future work will focus on improving the system's efficiency.
  • Automatic Disambiguation: The paper suggests incorporating automatic disambiguation for captioners, reducing the need for manual sense selection.
  • Corpus Specificity: Investigating whether specialized corpora (like photo captions) perform better for sense-tagging than general-purpose corpora (like the Brown corpus) is a future direction.
  • Integration with Existing IR Packages: Determining how to incorporate Intermezzo's NLP capabilities into existing IR systems is also a goal.

Summary updated

BibTeX
@inproceedings{flank1998layered,
  author = {Sharon Flank},
  title = {{A Layered Approach to NLP-Based Information Retrieval}},
  booktitle = {International Conference on Computational Linguistics},
  year = {1998},
}
Figure