SKG: A Versatile Information Retrieval and Analysis Framework for Academic Papers with Semantic Knowledge Graphs

Yamei Tu, Rui Qiu, Han-Wei ShenarXiv 2023 · DOI · arXiv

Introduction

In the current era of the "paper explosion," researchers in rapidly evolving fields like Artificial Intelligence and Data Visualization face a daunting challenge. With thousands of papers published annually, standard search tools like Google Scholar often provide overwhelming, flat lists of documents that require significant manual effort to synthesize. To address this, researchers have developed SKG, a versatile framework that leverages a Semantic Knowledge Graph and an intuitive visual dataflow interface to automate and enhance the literature review process.

Semantic Knowledge Graph Construction

The core innovation of SKG lies in its ability to understand the roles that concepts play within research abstracts. Unlike traditional Knowledge Graphs (KGs) that focus primarily on metadata—such as authors, citations, and publication years—SKG extracts semantic entities and categorizes them into five specific roles: Application, Data, Method, Visualization, and Evaluation.

The SKG ontology categorizes entities into metadata and semantic roles such as Application, Data, and Method.

To build this graph without the prohibitive cost of manual labeling, the authors employed a semi-supervised "weak supervision" approach. They utilized a RoBERTa-based Question-Answering (QA) model to automatically identify entities by asking targeted questions (e.g., "What is the method used in this study?"). These extracted concepts are then normalized using DBpedia, ensuring that synonymous terms like "Neural Net" and "Artificial Neural Network" are correctly merged into a single node.

Visual Dataflow System

To make this complex graph accessible to researchers without expertise in query languages like SPARQL, SKG introduces a visual dataflow system. Users can construct sophisticated analysis pipelines by dragging and dropping modular components. These are divided into two categories:

  • Operator Components: These include tools for querying nodes, expanding connections, and comparing different sub-graphs.
  • Visualizer Components: These provide graphical representations of the data, such as Sankey diagrams for flow analysis and node-link graphs for structural exploration.
The SKG visual interface enables the construction of complex analysis pipelines through modular operator and visualizer components.

This modular approach allows researchers to calculate relevance scores based on the number of "simple paths" between entities, surfacing non-obvious relationships that traditional keyword searches might miss.

The VISBank Dataset and Evaluation

The framework was evaluated using VISBank, a new large-scale dataset specifically curated for the visualization domain. Derived from the Semantic Scholar Open Research Corpus (S2ORC), VISBank contains 125,745 papers from over 3,500 venues. This dataset significantly expands upon previous benchmarks by including full citation relationships and a broader range of sub-fields.

Experimental results demonstrate the framework's efficacy:

  • Model Accuracy: The RoBERTa-based extraction model achieved high F1-scores between 0.8 and 0.9 for semantic concept identification.
  • Expert Alignment: In a case study involving a "State of the Art" (STAR) paper, the SKG automatically captured over 80% of the concepts identified by human experts.
  • Efficiency: The system successfully replicated complex co-authorship and topic networks that previously required manual curation, serving as a powerful "force multiplier" for domain experts.

Conclusion and Future Work

While currently optimized for the visualization domain and limited by its reliance on paper abstracts, SKG represents a significant step toward automated research synthesis. By recognizing the contextual roles of concepts, it provides a deeper understanding of the research landscape than metadata-only systems. Future iterations may integrate Graph Neural Networks (GNNs) to enable advanced features such as automated link prediction and personalized paper recommendations, further helping researchers navigate the ever-growing tide of scientific literature.

Summary updated

BibTeX
@misc{tu2023skg,
  author = {Yamei Tu and Rui Qiu and Han-Wei Shen},
  title = {{SKG: A Versatile Information Retrieval and Analysis Framework for Academic Papers with Semantic Knowledge Graphs}},
  year = {2023},
  eprint = {2306.04758},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2306.04758},
}
Figure