Abstract
Thanks to the rapid development of deep learning, big data analysis technology is not only widely used in the field of natural language processing, but also more mature in the field of numerical prediction. It is of great significance for the subject heat prediction and analysis of science and technology demand data. How to apply theme features to accurately predict the theme heat of science and technology demand is the core to solve this problem. In this paper, a prediction method of subject heat of science and technology demand based on time convolution network (TCN) is proposed to obtain the subject feature representation of science and technology demand. Time series prediction is carried out based on TCN network and self-attention mechanism, which increases the accuracy of subject heat prediction of science and technology demand data. Experiments show that the prediction accuracy of this algorithm is better than other time series prediction methods on the real science and technology demand datasets.
Introduction
In the rapidly evolving landscape of Research and Development (R&D), identifying which technology topics are gaining traction is crucial for strategic decision-making. "Topic heat"—the intensity of interest in a specific subject over time—serves as a vital metric for researchers and policymakers. However, predicting this heat is challenging due to the inherent complexity of science and technology (S&T) demand data, which often exhibits non-linear trends, seasonality, and random fluctuations.
Traditional models like ARIMA struggle with non-stationary data, while standard Recurrent Neural Networks (RNNs) often fail to capture long-range dependencies effectively. This paper proposes the SHDP-TCN framework to address these limitations by integrating thematic features with advanced deep learning architectures.
Methodology: The SHDP-TCN Framework
The proposed SHDP-TCN (Science and Technology Heat Demand Prediction - TCN) model utilizes three core components to enhance prediction accuracy:
- Self-Attention Mechanism: Acting as a pre-processor, this layer weights historical time steps based on their relevance to the current prediction, highlighting local correlations within the sequence.
- Temporal Convolutional Network (TCN): By employing causal and dilated convolutions, the TCN achieves a large receptive field, allowing it to process long historical sequences in parallel without information leakage from the future.
- Thematic Feature Integration: The model incorporates semantic category data (topic word features) to provide industry-specific context, strengthening the predictive power beyond simple numerical trends.
Experimental Setup and Results
The study utilized a dataset of 30,872 technology demand records spanning from 2012 to 2021, aggregated at monthly intervals. The data was categorized into 10 industry sectors, including Manufacturing, Agriculture, and Electronic Information. The model was trained on data prior to June 2019 and tested on the subsequent period.
Comparative analysis against baseline models demonstrates the superiority of SHDP-TCN, particularly when thematic features are included:
| Model | Precision | Recall | F1-Score |
|---|---|---|---|
| ARIMA | 0.4738 | 0.3036 | 0.3701 |
| LSTM | 0.5601 | 0.5064 | 0.5319 |
| CNN | 0.5417 | 0.5127 | 0.5268 |
| TCN (Standard) | 0.6125 | 0.4792 | 0.5377 |
| SHDP-TCN (Ours) | 0.7002 | 0.6903 | 0.6952 |
The SHDP-TCN achieved a precision of 0.7002, significantly outperforming traditional time-series methods and standard deep learning architectures. The results highlight that the "what" of a technology (thematic context) is just as important as the "when" (historical trend).
Conclusion
The integration of self-attention and thematic features within a TCN framework provides a robust solution for predicting the heat of science and technology topics. Future research will focus on increasing data granularity and incorporating external variables, such as policy shifts and economic indicators, to further refine prediction accuracy.
Summary updated