TL;DR
Lycos Retriever is an information fusion engine that automatically generates coherent, narrative topical summaries of popular web queries. Instead of just returning links, it synthesizes information from multiple sources into a browsable, Wikipedia-like article, aiming to provide users with a quick understanding of a topic.
What Problem Does This Paper Address?
The paper addresses the limitations of traditional search engine result pages (SERPs), which primarily present ranked lists of document links. While alternative interfaces like graphical maps or visualizations exist, they haven't gained widespread adoption. Question-answering interfaces also struggle with low adoption rates, partly due to users' preference for short, topic-based queries and the difficulty of reliably outperforming keyword search. The authors argue that users are increasingly responsive to curated, filtered information, as evidenced by the popularity of platforms like Wikipedia and Answers.com. They highlight the problem of "spam blogs" that repackage content without adding value, and the difficulty users face in sifting through potentially inflated search rankings.
Methods
Lycos Retriever employs a multi-stage process:
- Topic Selection: The system identifies popular web query topics.
- Categorization: A naïve Bayes classifier, trained on the DMOZ hierarchy, assigns categories to topics. Heuristics are used to ensure category uniformity and readability.
- Disambiguation: A disambiguator module determines if assigned categories refer to a single entity. It uses features learned from DMOZ data to cluster categories and assign discriminative terms, allowing for disambiguation (e.g., distinguishing "Saturn" the planet from "Saturn" the car brand).
- Document Retrieval: Up to 1000 documents are retrieved from Lycos' search provider based on the disambiguated topic. Optional "boost" terms are used for ambiguous topics, and terms from other senses are prohibited. Non-English and obscene pages are filtered.
- Passage Extraction: HTML documents are parsed into text blocks. Passages are identified as potentially interesting if they contain an expression of the topic in the first sentence. Further parsing verifies if the topic is the "Discourse Topic" of the passage, using patterns based on the CMU Link Parser.
- Passage Scoring: Passages are scored based on intrinsic informativeness metrics like paragraph length, unique NPs, and the verb associated with the Discourse Topic.
- Image Association: Images are thumbnailed and associated with paragraphs based on matching text in filenames, alt-text, descriptions, and proximity.
- Subtopic Selection and Report Assembly:
- Duplicate and near-duplicate paragraphs are removed.
- An "overview" section is populated with the best-scoring paragraphs, ordered by their role (defining the topic, describing its actions, or its outcomes).
- Subtopics are identified by clustering paragraphs based on the most frequent NPs they contain, with special handling for superstrings of the topic and non-reduced personal names.
- Paragraphs within subtopics are ordered similarly to the overview.
- Constraints are applied to respect fair use of copyrighted material.
- Topic reports are refreshed periodically.
Datasets
- DMOZ Hierarchy: Used for training the naïve Bayes classifier for topic categorization.
- Lycos Query Logs: A primary source for identifying popular query topics.
- Nationmaster.com, espn.go.com, curehodgkins.com, sportsline.com: Cited as sources for the "Mario Lemieux" example page.
- CMU Link Parser: Used for discourse topic identification.
No new benchmarks were introduced; the system was evaluated against existing tasks and datasets (e.g., KDD Cup).
Key Insights and Ideas
- Beyond Ranked Lists: The core idea is that users often need synthesized information, not just links. This moves beyond the traditional search paradigm to provide direct answers and context.
- Information Fusion for Summarization: The system effectively fuses information from disparate web sources into a coherent narrative, mimicking a human-curated summary.
- The Power of Disambiguation: Accurately categorizing and disambiguating topics is crucial for retrieving relevant documents and generating meaningful summaries. The use of DMOZ and learned features for disambiguation is a key component.
- Discourse Topic as a Relevance Signal: Identifying the "Discourse Topic" within a passage provides a strong signal for its relevance to the overall subject, going beyond simple keyword matching.
- User Responsiveness to Curated Content: The paper implicitly argues that the success of platforms like Wikipedia validates the need for systems that organize and present information in a structured, digestible format.
Key Results
- Categorization Precision: The live system achieved 86.5% precision in assigning non-default categories. A revised algorithm reached 93.0% precision on an evaluation of 982 topics.
- Unambiguous Topic Identification: Precision on identifying unambiguous topics with DMOZ was 83%, which compares favorably to the 75% precision of the best-performing system on a similar task in the 2005 KDD Cup.
- Initial Run: An initial run of approximately 60,000 topics yielded about 30,000 Retriever topic pages.
- Relevance: In an evaluation of 642 paragraphs, 88.8% were relevant to the topic, and 83.4% were relevant to the topic as categorized. For images, 85.5% of 83 images were relevant (using a revised algorithm). Of 1861 subtopic paragraphs, 88.5% were relevant to the assigned topic and subtopic.
Limitations and Future Work
- Temporal Ordering: The system did not pay attention to temporal ordering of paragraphs, leading to poorer performance on highly temporal topics where event precedence is important.
- Non-HTML Documents: The system was limited to extracting content from HTML documents and did not handle other formats.
- Auto-Hyperlinking: The authors suggest auto-hyperlinking topics within Retriever pages, similar to Wikipedia.
- Up-to-Date Sources: Finding more up-to-date sources for categorization was identified as a need.
- Verticalized Generation: The paper suggests verticalizing page generation for different topic types (e.g., movies, people, diseases) to tailor the output.
- Project Discontinuation: Unfortunately, the project was essentially discontinued in February 2006.
Summary updated