Browsing Help for a Faster Retrieval

Eric Crestan, Claude de LoupyCOLING 2004

TL;DR

This paper introduces "Intuition," a search engine that goes beyond keyword matching by offering "browse help" features like conceptualization and named entity navigation. Evaluations show these features significantly reduce search time and improve user satisfaction by helping users find relevant information more efficiently.

What Problem Does This Paper Address?

Traditional search engines, despite advancements, often struggle to meet user expectations for finding relevant information quickly. While metrics like recall and precision have improved, user satisfaction hasn't kept pace. The authors argue that the time spent by users to find answers is a critical, often overlooked, metric. They aim to address this by developing and evaluating features that assist users in navigating search results, rather than just presenting a flat list of documents.

Methods

The core of the paper is the "Intuition" search engine, which combines statistical analysis with linguistic knowledge. Key features include:

  • Conceptualization: This feature extracts short noun phrases (concepts) from the top search results. Clicking on these concepts refines the search, allowing users to explore related topics and narrow down results.
  • Named Entity Navigation: Intuition identifies and categorizes named entities (locations, people, organizations) within documents. Users can then filter search results based on these entities, similar to using metadata.
  • Named Entity Visualization: Identified named entities are highlighted within the document text, allowing users to quickly scan for relevant information and passages.
  • Similar Documents: This feature suggests up to three similar documents for any given document, aiding exploration.

The evaluation involved six interfaces, each offering a different combination of these features, tested with 18 queries on a corpus of 16 years of the French newspaper Le Monde. User actions were logged, and performance was measured by time to find the first relevant document, number of relevant documents retrieved, and average recall.

Datasets

  • Corpus: 16 years (1989-2002) of the French newspaper Le Monde (775,000 articles).
  • Queries: 18 queries per interface, comprising:
  • 12 task description queries from the TREC-6 ad-hoc campaign.
  • 6 factual questions from TREC Question/Answering evaluation campaigns.

Key Insights and Ideas

  • Beyond Keywords: The paper strongly advocates for moving beyond simple keyword matching in search. Linguistic analysis and user-centric navigation features are crucial for improving the user experience.
  • Time is of the Essence: User satisfaction is directly tied to the time it takes to find relevant information. Features that reduce this time are highly valuable.
  • Conceptual Navigation is Powerful: The "conceptualization" feature, allowing users to navigate via extracted noun phrases, proved to be a very effective way to refine searches and explore topics.
  • Named Entities as Filters: Using named entities as filters significantly helps users narrow down results and quickly identify documents relevant to specific people, places, or organizations.
  • Integrated Features Win: The interface incorporating all proposed features (Interface6) consistently outperformed others across most metrics, highlighting the synergistic effect of these browsing aids.

Key Results

  • Reduced Search Time: Interface6, with all features, achieved the shortest mean time to find the first relevant document (121.8 seconds), significantly outperforming the baseline classical search (Interface1 at 248.0 seconds).
  • Increased Relevant Documents: Interface6 also retrieved the highest average number of relevant documents (6.56 per query/user), nearly double that of Interface1.
  • Improved Recall: The average recall curves showed that interfaces with browsing features, particularly entity filtering and all features combined, achieved higher recall over time compared to the baseline.
  • Filtering Non-Relevant Documents: Features like conceptualization and named entity navigation helped users filter out non-relevant documents more effectively.

Limitations and Future Work

  • Small User Group: The evaluation was conducted with only six users, which the authors acknowledge is a limitation for evaluating six distinct interfaces. They plan to reproduce the evaluation with more users.
  • User Familiarity: Most users were familiar with the search engine and its features, which might have biased the results. Future work could involve training users before evaluation.
  • Extensive Testing Needed: While Interface3 showed a surprising initial rise over Interface6 in average recall, the authors suggest more extensive tests are needed to corroborate this.
  • Linguistic Analysis Value: The authors believe that linguistic knowledge and treatments, though sometimes controversial in classical IR, offer the easiest way to interact with users and improve search.

Summary updated

BibTeX
@inproceedings{crestan2004browsing,
  author = {Eric Crestan and Claude de Loupy},
  title = {{Browsing Help for a Faster Retrieval}},
  booktitle = {International Conference on Computational Linguistics},
  year = {2004},
}
Figure