TL;DR
This paper introduces a "2-step retrieval" method for Case-Based Reasoning (CBR) systems. By performing an initial broad retrieval and then refining the query with information from those results, the system significantly improves the accuracy and diversity of retrieved results, leading to better adaptation quality. This is particularly beneficial in complex domains like travel medicine where information is often incomplete.
What Problem Does This Paper Address?
Intelligent information systems, especially those using Case-Based Reasoning (CBR), rely heavily on effective knowledge acquisition, storage, and retrieval. A critical challenge in CBR is retrieving relevant cases and then adapting them to solve new problems. When dealing with complex domains like travel medicine, where information is vast, interdisciplinary, and often incomplete (e.g., disease prevalence changes), a single retrieval step might not yield sufficiently accurate or diverse results for effective adaptation. This can lead to missed information or the inclusion of irrelevant details. The paper addresses the problem of improving the accuracy and diversity of retrieved results in CBR systems to enhance the quality of case adaptation, particularly in domains with incomplete information.
Methods
The core contribution is the 2-step retrieval mechanism. This iterative approach involves:
- First Retrieval: An initial query is performed on the case base.
- Information Extraction: Attributes and similarity information from the cases returned in the first retrieval are analyzed.
- Second Retrieval: A refined query is constructed using the extracted information from the first step. This new query is then executed on the same case base.
The underlying idea is that the first retrieval, while potentially broad, provides valuable context and similarity information that can be used to create a more focused and accurate second query. This process aims to produce a more diverse distribution of similarities among the retrieved results, narrowing down the set of candidates for adaptation and improving the overall quality of the adapted solution. The authors implement this in the docQuery system for travel medicine, using a modularized knowledge base.
Datasets
The paper uses a custom-built case base for the docQuery project, which focuses on travel medicine. This case base covers information on countries, associated health risks, vaccinations, and travel advice. For evaluation, they created a controlled sample of 18 countries from East and South East Asia, manually filling in data on transmittable diseases to ensure complete information for these cases. This dataset is described as representative of country borders, coasts, islands, and climatic conditions.
Key Insights and Ideas
- Iterative Refinement is Key: The paper demonstrates that a single retrieval is often insufficient for complex domains. Iteratively refining queries based on initial results can dramatically improve the outcome.
- Diversity Enhances Adaptation: The 2-step retrieval doesn't just aim for more relevant results, but more diverse results. This diversity is crucial for finding better adaptation candidates when initial information is incomplete.
- Domain-Specific Attributes Matter: In travel medicine, attributes like vaccination requirements and disease prevalence are highly informative for refining queries, even if the initial query is just based on the destination country.
- Modularization Aids Flexibility: The
docQuerysystem's modularized case bases (e.g., for destinations, associated information) allow for flexible querying and combination of information, which is essential for the 2-step approach.
Key Results
The evaluation, conducted using a leave-one-out experiment on the travel medicine dataset, showed significant improvements:
- Reduced Adaptation Candidates: The number of potential adaptation candidates dropped significantly (by an average of 3.3 countries) in 90% of test cases when using 2-step retrieval compared to 1-step retrieval.
- Improved Accuracy (Fewer False Negatives): In 76% of cases, the adapted results from the 2-step retrieval contained all expected diseases, compared to only 62% for the 1-step retrieval. This means fewer diseases were missed.
- More Reliable Results: While both methods produced some false positives, the 2-step retrieval was generally more reliable, particularly in reducing false negatives.
- Better Adaptation Candidates: The paper highlights that the specific adaptation candidates selected by the 2-step retrieval were often more appropriate (e.g., Hong Kong being a better adaptation source for Japan than Mongolia).
Limitations and Future Work
Limitations:
- The paper acknowledges that the current system randomly picks one of the best cases as an adaptation source. They plan to incorporate maintenance information to select the most updated or recently maintained adaptation candidate in the future.
- While the system aims to provide complete information, it admits that for some aspects (like diseases contracted during a journey), information might still be incomplete and rely on adaptation from similar countries.
Future Work:
- Combining Modularized Case Bases: The authors plan to enable the system to combine retrieval results from different modularized case bases (e.g., activities, diseases, medication) to create comprehensive travel leaflets.
- Multi-Agent System: They intend to implement a multi-agent system with a broker agent to combine retrieval results and ensure completeness under given constraints.
- Generalization and Domain Applicability: A significant future direction is to generalize the 2-step retrieval algorithm and evaluate its applicability to other case bases and domains beyond travel medicine. They also want to explore if this query refinement strategy can be applied to other CBR systems.
Summary updated