Non-linear correlation analysis between internet searches and epidemic trends

Yu He, Lingshi Ran, Yang Wang, F. Huang, Yixue XiaFrontiers in Public Health 2025 · DOI

TL;DR

This paper introduces a novel non-linear differential equation model to quantify the dynamic, mutually reinforcing relationship between public internet search behavior (SI) and confirmed COVID-19 cases (CC). By analyzing the rates of change rather than just static values, the model reveals a stable feedback loop where increased searches predict case growth, and rising cases drive more searches, offering a more robust approach to epidemic monitoring.

What Problem Does This Paper Address?

Traditional methods for monitoring infectious disease outbreaks often struggle with the speed and complexity of real-time data. Internet search data offers a promising, immediate proxy for public interest and behavior, but previous studies have largely relied on simple linear correlations or descriptive analyses. These approaches often fail to capture the intricate, dynamic interplay between public information-seeking and disease spread, limiting their predictive power and ability to uncover underlying mechanisms. This paper aims to bridge this gap by developing a more sophisticated, non-linear framework that quantifies this dynamic interaction over time and across different epidemic phases.

Methods

The core of this research is a non-linear differential equation model built upon the logistic growth model. Instead of just fitting parameters to static data, the authors focus on analyzing the rates of change of both search index (SI) and confirmed cases (CC) over time. They employ a derivative-based approach to model the mutual influence, treating the interaction as a feedback loop.

Key methodological steps include:

  • Data Pre-processing: Crawling daily SI data from Baidu Index and daily CC data from China's National Health Commission. Initial Pearson correlation analysis showed a weak linear relationship, prompting the use of cumulative data.
  • Cumulative Data Analysis: Accumulating SI and CC data to better capture the "evolution process" and "temporality" of the data, revealing a stronger correlation.
  • Data Segmentation: Dividing the data into three yearly segments (2020, 2021, 2022) to account for potential shifts in epidemic dynamics and public behavior, which further improved correlation.
  • Building the Basic Model: Formulating logistic growth equations for SI and CC, incorporating terms for natural growth and upper limits, and then extending them to include impact functions ($f_1$ and $f_2$) representing the mutual influence.
  • Dynamic Regression Analysis: Using an incremental approach where data points are added sequentially to validate the model's structure and parameters across different data coverage rates (70%, 80%, 90%). This method quantifies the Conformity Rate (CR), indicating how well a structure fits the data across tests.
  • Impact Function Identification: Testing numerous combinations of terms for the impact functions ($f_1, f_2$) to identify common structures that hold across different epidemic phases.
  • Model Refinement: Introducing adjustment terms ($\delta_i, d_i$) to the impact functions to enhance model flexibility and accuracy in simulating real-world dynamics.

Datasets

  • Baidu Index: Used to collect daily search frequency data for the keyword "COVID-19" in China. This platform was chosen for its high market share and representativeness of Chinese internet user behavior.
  • National Health Commission of the People's Republic of China: Official website used to obtain daily confirmed COVID-19 case data for China. This source is considered authoritative and rigorous.

The study utilized data spanning from February 29, 2020, to December 23, 2022, covering a significant portion of the pandemic's evolution in China.

Key Insights and Ideas

  • Beyond Linear Correlation: The study strongly emphasizes that simple linear correlations are insufficient to capture the complex, dynamic relationship between public search behavior and epidemic spread. The non-linear, rate-of-change approach is crucial.
  • Mutually Reinforcing Feedback Loop: The most significant insight is the identification of a strong, stable feedback loop: increased public search interest (SI) directly drives the growth of confirmed cases (CC), and conversely, rising CC numbers stimulate further public information-seeking.
  • Epidemic Phases Matter: The relationship between SI and CC is not static. Segmenting the data by year (representing different epidemic phases) revealed stronger correlations and allowed for a more nuanced understanding of the dynamics.
  • Stable Structural Mechanisms: The model's focus on identifying common structural components of the impact functions across different epidemic phases suggests underlying, stable mechanisms that can be generalized.
  • Search Data as a Predictive Tool: The findings reinforce the idea that internet search data is not just a reflection of public sentiment but a valuable, real-time indicator for predicting epidemic evolution.

Key Results

  • High Correlation with Cumulative Data: After processing cumulative data, the correlation coefficient between SI and CC reached a significant 0.89, and further improved to 0.98 after segmenting the data by year.
  • Robust Model Fitting: Dynamic regression analysis showed high Conformity Rates (CR) for the identified model structures, with many segments achieving 95-100% CR, indicating excellent fit and stability.
  • Impact Function Structure: The study identified common structural components in the impact functions, such as terms like $a_i(x_i + \delta_i)x_j$ and $b_i(x_j + d_i)x_i$, which represent the interaction between SI and CC.
  • Promoting Effects: Heatmap analysis of the impact functions indicated generally promoting effects of CC on SI and vice versa, supporting the feedback loop hypothesis.

Limitations and Future Work

  • Personalization and Filter Bubbles: The authors acknowledge that search engine algorithms and personalized results can introduce bias and affect the generalizability of data. Future work could involve segmenting data by user characteristics or assigning differentiated weights.
  • Reporting Delays: Confirmed case data can be influenced by reporting timeliness and delays, especially during peaks. Integrating multi-source data (e.g., from authoritative hospital centers) is suggested to improve accuracy.
  • Limited Temporal and Geographic Scope: The current study primarily focuses on temporal dynamics and uses data from China. Future research should explore geographic differences and incorporate emotional responses (anxiety, uncertainty) more explicitly. Expanding to cross-country comparative analysis with similar data sources is also recommended.
  • Model Complexity: While the model is designed to be generalizable, the authors note that the underlying principles are not immediately obvious from the equations and require in-depth analysis.

Summary updated

Abstract

This study enriches the understanding of epidemic transmission mechanisms by quantifying the dynamic interaction between public search behavior and epidemic spread. Compared to simple prediction models, this study focuses more on stable common mechanisms and structural analysis, laying a foundation for future research on public health events.
BibTeX
@inproceedings{he2025non,
  author = {Yu He and Lingshi Ran and Yang Wang and F. Huang and Yixue Xia},
  title = {{Non-linear correlation analysis between internet searches and epidemic trends}},
  booktitle = {Frontiers in Public Health},
  year = {2025},
  doi = {10.3389/fpubh.2025.1435513},
}
Figure