TL;DR
ScholarStack introduces a layered framework that compiles scientific literature into reusable, versioned, and provenance-preserving knowledge assets. By breaking the bottleneck of AI agents repeatedly re-reading and re-interpreting the same papers for different tasks, it dramatically improves cross-paper reasoning while slashing query-time token costs by up to 76%.
What Problem Does This Paper Address?
Current LLM-based scientific agents are highly capable at isolated tasks like answering a question, summarizing a paper, or conducting a literature search. However, they suffer from a task-specific research bottleneck: they treat every request as an independent problem.
When an agent moves from searching for papers to writing a literature review, and then to designing an experiment, it repeatedly retrieves, segments, and interprets the exact same documents. The intermediate understandings it develops—such as aligned terminology, cross-paper relationships, and verified findings—are trapped in temporary context windows or task-specific reports. This lack of memory forces subsequent workflows to completely reconstruct the reasoning chain from scratch.
Methods
ScholarStack solves this by organizing scientific literature into a persistent, three-layer knowledge infrastructure ($K = (F, O, S)$):
- L1: Source-Grounded Scientific Facts ($F$): Anchors specific scientific statements directly to source sentences. It captures the precise context, assumptions, and study conditions (e.g., sample sizes, constraints) under which a finding holds true.
- L2: Domain Metadata and Organization ($O$): Builds a structured domain taxonomy. It maps canonical entities to resolve naming variations across papers, assigns categories, and establishes typed relations (e.g., "Method A improves over Method B").
- L3: Scoped Cross-Paper Syntheses ($S$): Integrates findings across multiple papers, explicitly declaring the scope of evidence and tracking supporting, opposing, or limiting claims.
Agents interact with this infrastructure through a unified access interface using five core semantic operations to generate task-specific "Knowledge Views":
Datasets
The authors evaluated ScholarStack across 4 task families spanning 10 distinct research settings using a mix of established benchmarks and newly introduced cross-domain datasets:
- Literature Retrieval: LitSearch (fixed-corpus), SAGE (open-world), and a newly built cross-domain benchmark for Novelty Assessment spanning 12 non-CS fields.
- Scientific Question Answering: QASPER and PeerQA for single-paper QA; MDAQA for multi-paper QA.
- Evidence-Grounded Generation: OARelatedWork (related-work generation), ReportBench-ML (literature review generation), a newly constructed cross-domain benchmark for Experimental Design, and IdeaBench / MOOSE-Chem / AI Idea Bench 2025 (idea generation).
- Claim Assessment: NLPCC 2026 Shared Task 10 (Track 2) for scientific claim verification.
Key Insights and Ideas
- Fidelity vs. Synthesis: By separating raw source facts (L1) from domain organization (L2) and cross-paper interpretations (L3), the framework allows agents to maintain strict source traceability. An agent can reuse a high-level synthesis without losing the ability to drill down into the exact study conditions that validate it.
- Amortizing the "Reading Tax": Compiling papers into structured assets once allows multiple downstream tasks to share the infrastructure. This shifts AI research assistance from isolated document processing to a cumulative workflow.
- Structured Context Beats Raw Text for Synthesis: LLMs struggle to synthesize information when flooded with raw, full-text papers due to context distraction. Presenting the model with pre-structured, aligned knowledge layers yields significantly more accurate and better-calibrated multi-paper reasoning.
Key Results
ScholarStack demonstrated substantial performance gains and token efficiencies across the board, with the most dramatic improvements occurring in tasks requiring cross-paper evidence synthesis:
- Multi-Paper QA (MDAQA): ScholarStack achieved an overall quality score of 18.41 vs. 8.46 for the full-text baseline (out of 20). Crucially, it did this while reducing generation token costs by 73.3% and cutting generation time nearly in half.
- Literature Review Generation (ReportBench-ML): It achieved the strongest expert-reference coverage, recovering 637 expert-survey references (0.214 F1 vs. 0.090 for Claude Code DeepResearch). It also achieved the highest report-quality score overall (9.18 out of 10).
- Claim Verification & Novelty Assessment: For scientific claim verification, query-time token usage dropped by 76.6% while maintaining a slight accuracy edge. In novelty assessment, token usage fell by 40% while improving the weighted identification F1 score.
Limitations and Future Work
While query-time token costs drop significantly, the framework introduces an upfront offline asset construction cost to parse and compile the layers. The paper does not provide a full end-to-end financial break-even analysis for this initialization phase.
Additionally, the authors note that future work must address how these assets evolve over time as new papers are published, requiring robust versioning and dynamic re-verification mechanisms when underlying source evidence changes.
Summary updated