Web Applications

IJCAI 2001

TL;DR

This paper introduces "Keyword Spices," a novel method for building domain-specific web search engines by augmenting user queries with automatically discovered keywords. This approach significantly enhances search precision and recall by leveraging existing general-purpose search engines without requiring manual domain expertise or extensive data collection.

What Problem Does This Paper Address?

The proliferation of web content has made information retrieval increasingly challenging. While general-purpose search engines are powerful, they often struggle to return highly relevant results for specific domains due to the vastness and heterogeneity of the web. Building domain-specific search engines traditionally involved creating specialized indices or filters, which were time-consuming, domain-dependent, and required significant human knowledge. This paper addresses the need for an efficient, scalable, and automated method to create effective domain-specific search engines that can improve the precision and recall of search results.

Methods

The core of this paper is the "keyword spice" method. Instead of building entirely new search engines or filters, this approach augments a user's initial query with domain-specific keywords (keyword spices). These spices are discovered using a machine learning algorithm, specifically a decision-tree learning algorithm. The process involves:

  1. Data Collection: Gathering web pages related to a specific domain (e.g., cooking recipes).
  2. Decision Tree Induction: Building a decision tree from this data to classify documents as relevant or irrelevant to the domain.
  3. Boolean Expression Conversion: Converting the decision tree into a Boolean expression, which represents the keyword spices.
  4. Simplification: Applying a two-stage simplification algorithm to reduce the complexity of the Boolean expression, making it suitable for commercial search engines.
  5. Query Augmentation: Appending the simplified keyword spices to the user's original query before submitting it to a general-purpose search engine.

This method effectively "spices up" the user's query, guiding the general search engine towards more relevant results within the target domain.

Datasets

The paper uses a dataset of 2000 sample web pages from the cooking domain. These pages were collected using common Japanese keywords like "beef," "chicken," "paprika," etc., and then manually classified as relevant or irrelevant. The dataset was split into training and validation sets for developing and refining the keyword spices.

Key Insights and Ideas

  • Augmentation over Replacement: The most significant insight is that enhancing existing search engine capabilities by augmenting queries is more practical and efficient than building entirely new systems.
  • Automated Keyword Discovery: The ability to automatically discover effective keyword spices using machine learning is a key innovation, removing the need for manual domain expertise.
  • Decision Trees for Boolean Expressions: The use of decision trees as an intermediate step to generate Boolean expressions is a clever way to bridge the gap between machine learning models and the query syntax of search engines.
  • Simplification is Crucial: The paper highlights the importance of simplifying complex Boolean expressions to ensure they are usable by commercial search engines and to avoid overfitting.
  • Precision-Recall Balance: The simplification algorithm's focus on maximizing the harmonic mean of precision and recall is a sound strategy for creating effective keyword spices.

Key Results

The experiments demonstrated significant improvements in search performance:

  • Precision: For queries with keyword spices, precision was consistently higher than 97% for the top 1000 returned results across tested keywords (pork, spinach, shrimp). This is a substantial improvement over queries using only the input keyword.
  • Recall: The estimated recall for spice-extended queries was higher than 87%, indicating that the method effectively filters out irrelevant documents without losing significant relevant information.
  • Efficiency: The keyword spice method is presented as a low-cost solution, as it leverages existing search engine infrastructure and requires less manual effort compared to traditional methods.

Limitations and Future Work

  • Initial Keyword Selection: The current method relies on human-selected input keywords for generating training examples. The authors acknowledge that a more comprehensive approach would involve criteria for automatically selecting input keywords, potentially based on user query distribution.
  • Future Work: The authors plan to open their recipe search system to the public to gather data on user query distributions and refine the keyword selection process. They also intend to apply the keyword spice method to other domains, such as restaurant pages and personal homepages.

Summary updated

BibTeX
@inproceedings{2001web,
  title = {{Web Applications}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2001},
}
Figure