TL;DR
This paper introduces SCISOR, a system designed to tackle the challenge of information retrieval from continuously unfolding news stories, specifically in the domain of corporate takeovers. By moving beyond traditional full-text retrieval, SCISOR uses a knowledge-based approach to understand and summarize evolving events, providing users with concise updates rather than entire documents.
What Problem Does This Paper Address?
The paper addresses a fundamental limitation of traditional information retrieval (IR) systems: their inadequacy for handling information that evolves over time and is presented in multiple, short, incremental updates. Current systems primarily retrieve entire documents, which is inefficient and often overwhelming when dealing with "never-ending stories" like corporate takeovers. Users often need specific facts or summaries of events, not just a collection of articles. The corporate takeover domain is particularly challenging because:
- Information is incremental: Each new article adds a small piece to an ongoing narrative.
- Redundancy is common: Articles frequently rehash past events, making it hard to extract only new information.
- Events are not self-contained: Stories unfold over long periods, requiring users to piece together information from numerous sources.
This makes it difficult for users to get quick answers, summaries, or updates without manually sifting through a large volume of text.
Methods
SCISOR employs a natural language, knowledge-based approach to information retrieval. Key components and techniques include:
- TRUMP (Transportable Understanding Mechanism Package) parser and semantic interpreter: Used to parse both news articles and user questions, enabling the system to understand the content.
- Knowledge Representation: Articles and questions are represented using the KODIAK knowledge representation language, augmented with scriptal knowledge of typical event sequences in the corporate takeover domain.
- Story Event Integration Mechanism: This component is crucial for "filling in" new information into existing story summaries and unifying references to past events found in new articles with previously stored representations. This allows the system to track the evolution of a story.
- Retrieval Mechanism: Operates using a form of constrained marker-passing to efficiently find answers to user questions or to retrieve the history of a story being updated. It focuses on identifying likely candidates for answers.
- Generalization Mechanism: Designed to identify new trends by automatically detecting multiple instances of similar situations.
- KING (Knowledge INtensive Generator) natural language generator: Used to express the system's answers in natural language.
The system aims to provide answers based on explicitly stored information and inferences derived from context and world knowledge, moving beyond simple keyword matching.
Datasets
The paper does not explicitly mention specific datasets used for training or evaluation in terms of large-scale benchmarks. Instead, it describes SCISOR being tested with "a dozen or so stories stored in the knowledge base." The domain itself, corporate takeovers, serves as the primary source of information.
Key Insights and Ideas
- The limitations of full-text retrieval are domain-dependent: While full-text search is powerful, its effectiveness diminishes significantly when dealing with evolving narratives and incremental information updates, as seen in the corporate takeover domain.
- Understanding is key to effective retrieval: Moving beyond simple document retrieval to conceptual understanding allows systems to provide more targeted and useful information, such as concise summaries or direct answers to questions.
- Tracking evolving events requires integration: A system needs to actively integrate new information with existing knowledge to maintain an accurate and up-to-date representation of an ongoing story.
- Predictability aids parsing: The predictable nature of events in constrained domains like corporate takeovers makes expectation-driven, partial parsing feasible, improving efficiency.
- Natural language interaction is crucial for user-friendliness: Allowing users to ask questions in natural language and receive direct answers or summaries significantly enhances the usability of information systems.
Key Results
The paper highlights the capabilities of SCISOR rather than presenting extensive quantitative results. The demonstrated capabilities include:
- Answering straightforward questions: e.g., "Did ACE hardware take over the ACME food company?" with a direct "Yes" and a brief explanation.
- Summarizing event sequences: e.g., providing a timeline of the ACE-ACME takeover deal, including rumors, offers, and completion.
- Providing updates on ongoing stories: e.g., reporting on a new bid in the HAL Corporation bid for BIG.
The system is implemented in Common Lisp and runs on VAX, Symbolics, and SUN workstations. It processes input at a rate of a few seconds per sentence.
Limitations and Future Work
- Limited scope of questions: Currently, SCISOR can only answer questions about information explicitly stored in its knowledge base. It cannot yet perform complex inferential reasoning or comparative analysis across multiple entities (e.g., "Which companies have been taken over more times than they have taken over other companies?").
- Testing scale: The system has not yet been tested on a large number of documents or a wide range of real-world information retrieval tasks.
- Future work: The authors suggest that the next stage of the project will involve testing the system with a very large sample of documents to assess its ultimate usefulness in real information retrieval tasks. They also imply that adding more sophisticated processing capabilities for complex queries is a future direction.
Summary updated