Rivendell: Project-Based Academic Search Engine

Teddy Lazebnik, Hanna Weitman, Yoav Goldberg, Gal A. KaminkaarXiv 2022 · DOI · arXiv

Abstract

Rivendell is a meta-search engine designed to mitigate "concept drift" in academic research by organizing searches into user-defined projects. By isolating search history to specific contexts, the system improves precision by 12.8% and significantly reduces the number of queries required to find relevant literature.

Introduction

Finding relevant literature is a persistent challenge for researchers. Existing search engines typically follow one of two paradigms: one-time search, which treats every query in isolation, or life-time search, which uses a user's entire history to personalize results. Both approaches suffer from concept drift. Researchers often work on multiple unrelated projects simultaneously; a life-time search engine may erroneously use physics search history to "personalize" a biology search, introducing noise. Conversely, one-time engines ignore valuable context. Rivendell bridges this gap through "project-based search," siloing search history within specific research contexts.

Methodology

Rivendell operates as a meta-search engine atop arXiv and PubMed. Its architecture emphasizes structural data handling over complex neural models to isolate the benefits of project-based organization.

Explicit Project Context

Users are required to create a "project" before initiating a search. This ensures that queries and results are siloed, preventing cross-contamination between different research areas.

A project card for 'information retrieval' tracking searches and relevant abstracts.

Search and Personalization

The system translates user queries into Boolean logic to interface with external APIs. Results are then ranked and filtered using:

  • Monge-Elkan algorithm (with Levenshtein distance) for query similarity.
  • Cosine similarity to compare new abstracts against those previously marked as "relevant" by the user.
Rivendell dashboard showing aggregated results from PubMed and arXiv within a specific project context.

Next Query Suggestion

Rivendell analyzes tagged abstracts to suggest refined query terms. This acts as a cognitive nudge, helping users converge on relevant results faster.

Evaluation and Results

The system was evaluated through a live user study involving 199 participants (primarily graduate students and researchers). Real-time data was pulled from PubMed and arXiv, and user logs were analyzed for precision and convergence speed.

Key findings include:

  • Precision Gains: Project-based search outperformed life-time search by 12.8% after just six queries.
  • Efficiency: The query suggestion system halved the number of queries needed to reach high precision.
  • The "Rethink" Effect: While users only directly followed suggestions 8% of the time, the presence of suggestions led to a 30% improvement in finding relevant results, as it prompted users to rethink their own search strategies.

Discussion and Conclusion

Rivendell demonstrates that explicit context significantly improves search relevance. While the current implementation uses simple NLP techniques and requires manual tagging, it provides a strong baseline for future integration with state-of-the-art models like BERT. Future work will explore "community learning" to leverage shared tags across multiple users in specific sub-fields.

Summary updated

BibTeX
@misc{lazebnik2022rivendell,
  author = {Teddy Lazebnik and Hanna Weitman and Yoav Goldberg and Gal A. Kaminka},
  title = {{Rivendell: Project-Based Academic Search Engine}},
  year = {2022},
  eprint = {2206.12926},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2206.12926},
}
Figure