Calculating Semantic Similarity between Academic Articles using Topic Event and Ontology

Ming Liu, Bo Lang, Zepeng Gu

Abstract: Determining semantic similarity between academic documents is crucial to many tasks such as plagiarism detection, automatic technical survey and semantic search. Current studies mostly focus on semantic similarity between concepts, sentences and short text fragments. However, document-level semantic matching is still based on statistical information in surface level, neglecting article structures and global semantic meanings, which may cause the deviation in document understanding. In this paper, we focus on the document-level semantic similarity issue for academic literatures with a novel method. We represent academic articles with topic events that utilize multiple information profiles, such as research purposes, methodologies and domains to integrally describe the research work, and calculate the similarity between topic events based on the domain ontology to acquire the semantic similarity between articles. Experiments show that our approach achieves significant performance compared to state-of-the-art methods.

Introduction

Measuring the similarity between long academic articles is a complex challenge. Traditional systems often rely on surface-level statistics, such as the Vector Space Model (VSM) or Latent Dirichlet Allocation (LDA). While these methods are effective for keyword matching or broad topic distribution, they frequently fail to capture the underlying "soul" of a research paper.

Two papers may share significant terminology but serve entirely different purposes—for instance, one might propose a novel algorithm while the other provides a survey of existing ones. Conversely, papers using different jargon may describe identical methodologies. To address this, we propose a framework that accounts for the inherent structure of research: its purpose, methods, and specific style.

The Topic Event (TE) Framework

The core innovation of this research is the Topic Event (TE) framework. Rather than treating a document as a flat string of text or a "bag of words," we extract a structured summary that captures the essential components of the research work.

The structured definition of a Topic Event (TE) for academic papers.

A Topic Event consists of several primary elements:

  • Target: The essential purpose or goal of the research.
  • Methodology: The specific techniques or algorithms employed.
  • Domain: The sub-field of study (e.g., Computational Linguistics).
  • Style: The manner of research, such as "System Implementation" or "Theoretical Origination."
  • Keywords: Essential descriptive terminologies.
  • Date: The publication timing, providing temporal context.

Methodology and System Architecture

Our approach integrates structured extraction with semantic knowledge resources to compute similarity. The system utilizes two key ontologies: a Research Style Ontology, which categorizes the "manner" of the paper, and a Domain Ontology, which maps deep conceptual relationships within the specific field.

Workflow for computing document semantic similarity using Topic Events and Ontologies.

The process involves three main stages:

  1. Extraction: Using an ontology-and-pattern-based approach, the system automatically identifies TE elements from the Title, Abstract, Introduction, and Conclusion. This method relies on "trigger words" and noun-phrase extraction, bypassing the need for massive labeled training sets.
  2. Semantic Expansion: Concepts extracted are mapped to the ontologies to account for synonyms and hierarchical relationships (e.g., recognizing that "NER" is a sub-field of "Information Extraction").
  3. Similarity Calculation: The final similarity score is a weighted sum of the similarities between individual TE elements. We employ the Wu and Palmer method to measure conceptual distance within the ontologies.

Experimental Results

We evaluated our approach using the ACL Anthology Network (AAN) and a newly annotated corpus of 1,021 paper pairs from the computational linguistics domain. Our Topic Event-based method was compared against a strong LDA baseline:

  • Performance: The AutoTE_Onto method achieved a 5.8% relative improvement in Pearson correlation over the baseline. The best F1-score reached 0.639, significantly higher than the baseline's 0.536.
  • Efficiency: Despite the structural complexity, our method is highly efficient. The average computation time per document pair was 0.002s, compared to 4.83s for the LDA-based measurement.

Conclusion

By shifting from statistical surface-level matching to structured "Topic Event" profiles, we can more accurately capture the semantic essence of academic articles. While the system currently depends on the availability of high-quality domain ontologies, it demonstrates that research structure—specifically the interplay between methodology, target, and style—is a superior metric for document-level similarity. Future work will explore automating ontology construction and tuning element weights for diverse academic disciplines.

Summary updated

Abstract

Determining semantic similarity between academic documents is crucial to many tasks such as plagiarism detection, automatic technical survey and semantic search. Current studies mostly focus on semantic similarity between concepts, sentences and short text fragments. However, document-level semantic matching is still based on statistical information in surface level, neglecting article structures and global semantic meanings, which may cause the deviation in document understanding. In this paper, we focus on the document-level semantic similarity issue for academic literatures with a novel method. We represent academic articles with topic events that utilize multiple information profiles, such as research purposes, methodologies and domains to integrally describe the research work, and calculate the similarity between topic events based on the domain ontology to acquire the semantic similarity between articles. Experiments show that our approach achieves significant performance compared to state-of-the-art methods.
BibTeX
@misc{liu2017calculating,
  author = {Ming Liu and Bo Lang and Zepeng Gu},
  title = {{Calculating Semantic Similarity between Academic Articles using Topic Event and Ontology}},
  year = {2017},
  eprint = {1711.11508},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1711.11508},
}
Figure