Academic Reader: An Interactive Question Answering System on Academic Literatures

Yining Hong, Jialu Wang, Yuting Jia, Weinan Zhang, Xinbing Wang

Introduction

The rapid expansion of scientific literature presents a significant challenge for researchers attempting to stay current with their fields. Manually scanning hundreds of paper abstracts to identify core contributions and methodologies is a time-consuming process. While Machine Reading Comprehension (MRC) has flourished in general domains, its application to specialized academic literature remains limited. Academic Reader addresses this gap by providing an interactive system that extracts precise technical details from abstracts in response to user queries.

System Architecture

Academic Reader employs a multi-stage neural network pipeline designed to handle the linguistic complexities of academic writing. The process begins with a "Context" (the abstract) and a "Query" (the user's question).

The multi-stage neural network architecture of Academic Reader, featuring BiDAF and biLSTM-CRF components.

The architecture consists of three primary stages:

  1. Query-Aware Representation: Using Bidirectional Attention Flow (BiDAF), the system generates a representation of the abstract that emphasizes information relevant to the specific query.
  2. Sentence Ranking: A Multi-Layer Perceptron (MLP) calculates a "match score" for each sentence within the abstract. This step identifies the "evidence sentence" most likely to contain the answer, reducing noise for the subsequent stage.
  3. Span Extraction: The identified evidence sentence is processed by a biLSTM-CRF (Conditional Random Field) model. This component performs sequence tagging to extract the exact span of words that constitutes the final answer.

Interactive Interface and Active Learning

A core innovation of Academic Reader is its "human-in-the-loop" design. The system is not merely a static extraction tool but an evolving model that learns from user interaction.

The Academic Reader interface, demonstrating the question-answering process and the revision mechanism for active learning.

The interface supports multiple input methods, including direct text entry, arXiv ID fetching, and batch processing. When the system generates an answer, users have the option to "Revise" it if the result is unsatisfactory. These user-provided corrections are fed back into the system via an active learning loop, allowing the model to refine its performance on specialized technical jargon and domain-specific structures over time.

Case Study and Results

The utility of the system was demonstrated using a dataset of papers from the KDD 2018 conference. The system was tasked with answering two fundamental questions for each paper: "What problem does this paper study?" and "What method/model/framework does this paper propose?".

The results indicated that Academic Reader could effectively condense complex abstracts into concise, two-sentence summaries. This capability allows researchers to rapidly filter through conference proceedings, focusing their attention on papers most relevant to their specific interests.

Conclusion and Future Directions

Academic Reader represents a significant step toward automated literature survey tools. By combining sentence-level ranking with word-level span extraction and an active learning mechanism, the system provides high-precision answers in a specialized domain. Future work will focus on expanding the system's scope beyond Computer Science to fields like physics and mathematics, as well as scaling the context window to process full-length papers and integrating external knowledge graphs for deeper semantic understanding.

Summary updated

Abstract

We present Academic Reader, a system which can read academic literatures and answer the relevant questions for researchers. Academic Reader leverages machine reading comprehension technique, which has been successfully applied in many fields but has not been involved in academic literature reading. An interactive platform is established to demonstrate the functions of Academic Reader. Pieces of academic literature and relevant questions are input to our system, which then outputs answers. The system can also gather users’ revised answers and perform active learning to continuously improve its performance. A case study is provided presenting the performance of our system on all papers accepted in KDD 2018, which demonstrates how our system facilitates massive academic literature reading.
BibTeX
@inproceedings{hong2019academic,
  author = {Yining Hong and Jialu Wang and Yuting Jia and Weinan Zhang and Xinbing Wang},
  title = {{Academic Reader: An Interactive Question Answering System on Academic Literatures}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2019},
  doi = {10.1609/aaai.v33i01.33019855},
}
Figure