Research topic trend prediction of scientific papers based on spatial enhancement and dynamic graph convolution network

Changwei Zheng, Zhe Xue, MeiYu Liang, Feifei KouarXiv 2022 · DOI · arXiv

Introduction

Predicting the evolution of scientific research topics is crucial for resource allocation in academia and industry. Researchers at the Beijing University of Posts and Telecommunications have introduced SDTGCN (Spatial Dependency enhanced Temporal Graph Convolution Network), a novel framework designed to forecast "hot" topics by modeling both inter-topic influences and temporal popularity trends. By combining Graph Convolutional Networks (GCNs) with Temporal Convolutional Networks (TCNs), the model outperforms traditional sequence-based methods in both accuracy and computational efficiency.

Problem Statement

Traditional forecasting models, such as ARIMA or LSTMs, often treat research topics as isolated time series. This approach ignores "spatial" dependencies—the intricate web of citations and interdisciplinary connections where a breakthrough in one field (e.g., Graph Neural Networks) triggers interest in another (e.g., Drug Discovery). Furthermore, recurrent models like LSTMs struggle with long-range dependencies and high computational costs due to sequential processing. The authors set out to build a model that captures both the where (spatial relationships) and the when (temporal trends) of scientific evolution.

Methodology: The SDTGCN Framework

The SDTGCN model addresses these challenges through a dual-architecture approach that captures the spatial and temporal dynamics of scientific literature.

Architecture of the SDTGCN model showing spatial and temporal feature extraction layers.
  1. Spatial Enhancement (GCN): Research topics are represented as nodes in a dynamic graph. Edges are weighted based on citation relationships, ensuring that highly influential topics exert a stronger influence on the network's representations. This "Spatial Feature Enhancement" uses citation counts as weights to refine the spatial dependency representation.
  2. Temporal Dynamics (TCN): Instead of recurrent units, the model employs Temporal Convolutional Networks with dilated convolutions. This allows for a vast receptive field without the computational overhead of RNNs, enabling parallel processing of time steps.
  3. Weighted Loss Optimization: To prioritize current trends, the loss function assigns higher weights to more recent data points during training, forcing the model to focus on the most relevant historical data.

Experimental Results

The model was evaluated using the DBLP dataset (1970–2019), focusing on 500 key research topics. The task was to predict topic "hotness" (paper count) for year $T+1$ based on a window of $T$ years.

Performance comparison of SDTGCN against baseline models across various metrics (MAE, MSE, VAR) and sequence lengths.

Superior Accuracy and Stability

As shown in the results table, SDTGCN consistently outperforms baselines like Historical Average (HA), LSTM, and other spatiotemporal models (A3TGCN, LRGCN). In an 8-year prediction task, SDTGCN reduced the Mean Absolute Error (MAE) by approximately 53% compared to the HA baseline. High variance (VAR) scores further indicate that the model effectively captures the underlying signal even over long durations.

Computational Efficiency

Beyond accuracy, SDTGCN demonstrates significant scalability. While RNN-based models experience steep increases in processing time as the input sequence grows, SDTGCN’s convolutional nature allows it to remain efficient, making it more suitable for large-scale trend analysis across thousands of research topics.

Discussion and Conclusion

A key insight from this research is that scientific graphs are inherently dynamic. SDTGCN captures this by treating the graph as a sequence of snapshots. However, the authors note a "smoothing" effect: because the model aggregates information from neighbors, it may under-predict sudden, extreme bursts of interest in specific topics (the "peak prediction" problem).

In conclusion, SDTGCN provides a robust and efficient solution for scientific trend prediction by integrating spatial enhancement with temporal convolutions. Future work will focus on improving the capture of "spiked" anomalies and exploring more granular citation types to further refine the spatial enhancement module.

Summary updated

Abstract

In recent years, with the increase of social investment in scientific research, the number of research results in various fields has increased significantly. Accurately and effectively predicting the trends of future research topics can help researchers discover future research hotspots. However, due to the increasingly close correlation between various research themes, there is a certain dependency relationship between a large number of research themes. Viewing a single research theme in isolation and using traditional sequence problem processing methods cannot effectively explore the spatial dependencies between these research themes. To simultaneously capture the spatial dependencies and temporal changes between research topics, we propose a deep neural network-based research topic hotness prediction algorithm, a spatiotemporal convolutional network model. Our model combines a graph convolutional neural network (GCN) and Temporal Convolutional Network (TCN), specifically, GCNs are used to learn the spatial dependencies of research topics a and use space dependence to strengthen spatial characteristics. TCN is used to learn the dynamics of research topics' trends. Optimization is based on the calculation of weighted losses based on time distance. Compared with the current mainstream sequence prediction models and similar spatiotemporal models on the paper datasets, experiments show that, in research topic prediction tasks, our model can effectively capture spatiotemporal relationships and the predictions outperform state-of-art baselines.
BibTeX
@misc{zheng2022research,
  author = {Changwei Zheng and Zhe Xue and MeiYu Liang and Feifei Kou},
  title = {{Research topic trend prediction of scientific papers based on spatial enhancement and dynamic graph convolution network}},
  year = {2022},
  eprint = {2203.16256},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2203.16256},
}
Figure