TL;DR
This paper introduces BIG, an AI agent designed to tackle the overwhelming task of information gathering on the World Wide Web. BIG integrates various AI techniques to intelligently plan, schedule, and execute information retrieval and extraction, aiming to provide users with synthesized, actionable information rather than just raw data.
What Problem Does This Paper Address?
The World Wide Web is a vast repository of information, but its sheer volume, diverse sources, and varying quality make it incredibly difficult for humans to effectively gather relevant data for decision-making. Traditional information retrieval (IR) and information extraction (IE) methods, while improving, are insufficient to handle the complexity and dynamic nature of the web. Users are faced with a daunting task of planning, filtering, and interpreting information, leading to potential decision paralysis or suboptimal choices. The paper argues that a more sophisticated, automated approach is needed to manage this "information explosion."
Methods
BIG employs a multi-component architecture that integrates several AI techniques:
- Task Assessor: Formulates initial information gathering plans and revises them based on new information. It reasons about trade-offs between quality, cost, and duration of different gathering approaches.
- Design-to-Criteria Scheduler: Uses TÆMS (a domain-independent task modeling framework) to create custom schedules for achieving high-level goals, balancing resource constraints and desired outcomes.
- RESUN Planner: A blackboard-based planner that directs information gathering and processing activities. It's designed to be opportunistic, working on the most fruitful aspects of the information gathering process as new evidence emerges.
- Web Retrieval Interface: A generic interface for interacting with various search engines and retrieving URLs.
- Information Extractors: A suite of tools (e.g.,
textext-ks,grep-ks,tablext-ks) that process unstructured, semi-structured, and structured documents to extract specific data. These extractors range from heavy-weight NLP systems to lightweight pattern matchers. - Object Database & Server Information Database: Stores learned information, discovered objects, and attributes of information sources to support future searches and improve efficiency.
The core novelty lies in the integration of these components to create a system that not only gathers information but also reasons about the process of gathering it, considering resource constraints and the quality of the information obtained. It recasts information gathering as an "interpretation problem," where low-level data is used to build high-level models, acknowledging that evidence can be incomplete or inconsistent.
Datasets
Not discussed in the paper. The paper focuses on the architecture and methodology, using hypothetical examples and describing the types of information sources (e.g., news agencies, digital libraries, review sites) that BIG would interact with.
Key Insights and Ideas
- Information Gathering as Interpretation: The paper frames web information gathering not just as retrieval, but as an interpretation task, akin to building models from uncertain and incomplete evidence.
- Resource-Bounded Reasoning is Crucial: The sheer scale of the web necessitates agents that can reason about trade-offs between time, cost, and the quality of information gathered. BIG explicitly models these trade-offs.
- Constructive Problem Solving: BIG builds up potential answers (e.g., product descriptions) incrementally, refining them as new information is acquired, mirroring human-like opportunistic search.
- Integration of Diverse AI Techniques: The paper highlights the power of combining planning, scheduling, text processing, and reasoning under uncertainty within a single agent.
- The Need for Higher-Level Interfaces: The authors argue that future information systems must move beyond simply retrieving documents to providing synthesized information that supports decision-making.
Key Results
The paper presents a conceptual sketch and a detailed example of BIG in action, demonstrating its ability to:
- Generate a plan for information gathering based on user-defined criteria (quality, time, cost).
- Schedule information gathering activities, balancing trade-offs (e.g., choosing a slightly longer search path for higher quality results).
- Retrieve and process documents using various information extraction techniques.
- Synthesize information from multiple sources to create product descriptions.
- Make a decision based on the gathered and interpreted information, providing a recommendation.
Figure 4 shows experimental results for a 10-minute search case, illustrating how different time allotments affect the number of products considered, information coverage, and ultimately, the quality of the decision. For instance, a 5-minute run might yield a suboptimal recommendation due to insufficient information, while longer runs improve the quality and accuracy of the chosen product. The paper also notes that using previously learned information can significantly improve recommendation quality even within a short timeframe.
Limitations and Future Work
- Integration of RESUN Planner: While components are integrated and tested, the full opportunistic capabilities of the RESUN planner are not yet completely integrated. Specifically, the system doesn't yet dynamically replan top-down strategies based on new, unexpected information discovered during the search.
- Domain Dependency: While some components are domain-independent, others (like information extractors) require domain-specific training. Adapting BIG to a new domain would necessitate retraining these components.
- Database Aging: The server and object databases store past experiences, but the issue of aging stored data and its potential to become outdated or distracting is acknowledged.
- Dynamic Replanning: Future work includes developing a two-way feedback loop between the planner, task assessor, and scheduler to enable more dynamic replanning and better reaction to changes in the problem-solving state.
- Anytime Process: The authors intend to study BIG's control as an "anytime process," allowing it to provide a useful result at any point, with quality improving over time.
- Modeling TÆMS: Enhancements to TÆMS are proposed to better model data-driven dependencies and non-local task interactions (hinders/facilitates).
Summary updated