Introduction
Information retrieval systems typically excel when users provide precise keywords or exact titles. However, human memory often fails in specific ways, leading to the "tip-of-the-tongue" (TOT) phenomenon. In these scenarios, a user might remember vague contextual details—such as a book's cover art, a general plot point, or the approximate time they read it—without recalling the title or author. These queries are characterized by high complexity and low lexical overlap with the target documents, making them difficult for standard search engines to resolve.
Methodology: The MOREL Framework
To address the challenges of TOT retrieval, this research introduces MOREL, a framework that treats retrieval as a task of multi-step decomposition and expert ensembling. The system operates in three primary stages:
- Query Decomposition: Utilizing a Large Language Model (LLM) like GPT-3.5, the system breaks down a long, conversational query into discrete "clues." These clues categorize information into specific facets such as visual descriptions of the cover, estimated publication dates, and potential (often incorrect) titles.
- Specialized Retrieval Experts: Each extracted clue is routed to a specialized model. A Cover Retriever uses CLIP to match text descriptions against a database of book covers. A Date Retriever applies symbolic logic to filter results by publication year. A Title Retriever (using Contriever) focuses on the user's guesses, while a Base Retriever processes the entire query against book descriptions.
- Ensembling: The scores from these diverse experts are aggregated using a weighted sum to produce a final ranked list of candidates.
The WhatsThatBook Dataset
The researchers developed a new benchmark, WhatsThatBook, consisting of 14,441 real-world query-book pairs sourced from the Goodreads "What's That Book?" community. This dataset is unique because it contains "solved" queries where community members successfully identified books based on the posters' vague descriptions. Each entry includes the original complex query, the target book's metadata, and its cover image, providing a rich ground truth for evaluating TOT retrieval systems.
Key Insights and Results
The evaluation focused on Recall@K, measuring how frequently the correct item appeared in the top results. Standard text-only baselines like BM25 struggled, achieving only 8.3% Recall@5. In contrast, the MOREL framework achieved 28.4% Recall@5, representing a significant improvement over holistic dense retrieval models like Contriever (26.5%).
Key findings include:
- Visual Memory Integration: Using CLIP to bridge textual descriptions and actual cover images proved highly effective, capturing books that text-only searches missed.
- LLMs as Logic Translators: LLMs successfully translated relative timeframes (e.g., "read in high school in 2002") into absolute publication date filters, effectively pruning the search space.
- Decomposition Advantage: Breaking queries into facets allowed the system to leverage the specific strengths of different models (e.g., semantic matching vs. image-text alignment).
Conclusion and Future Work
The MOREL framework demonstrates that decomposing complex, conversational queries into structured facets significantly enhances retrieval performance for "tip-of-the-tongue" searches. Future research may explore dynamic weighting of ensemble scores based on query characteristics and extending this methodology to other media domains such as film and music.
Summary updated