TL;DR
This paper introduces a novel Multi-Self-Regression Deep (MSRD) neural network model for early warning of infectious diseases in hospitals. By integrating hospital data with environmental and social factors, the MSRD model offers more accurate and timely predictions compared to traditional methods like SEIR, addressing limitations in data availability and model flexibility.
What Problem Does This Paper Address?
The timely detection and prediction of infectious disease outbreaks are crucial for effective public health interventions. While national surveillance systems exist, they often lack the granularity and speed needed for real-time hospital-level insights. Existing infectious disease modeling approaches, such as the SEIR model, face challenges due to the difficulty in obtaining precise quantitative data (e.g., latent populations), susceptibility to overfitting with long time series, and a tendency to focus on single data streams without considering complex interdependencies. This gap hinders hospitals from effectively deploying resources for epidemic prevention and control.
Methods
The core of this research is the Multi-Self-Regression Deep (MSRD) neural network. This model builds upon a recurrent neural network (RNN) architecture, specifically using Long Short-Term Memory (LSTM) units. The MSRD model employs a sliding window approach to process time-series data, allowing it to capture historical development characteristics.
Key methodological innovations include:
- Multi-Source Data Integration: The model incorporates not only daily reported infectious disease data from hospitals but also weather data (temperature, humidity, wind) and social factors (e.g., student holidays to represent group activities).
- Self-Regression Learning: It uses time-series data from multiple days as input to predict the next day's confirmed cases, effectively modeling epidemiological trends.
- Feature Engineering: The model leverages a rich set of features, including daily confirmed cases, current date (year, month, season), climate data (max temp, min temp, temp difference), and social activity indicators.
- Hybrid Architecture: The MSRD model combines the output of LSTM layers (capturing temporal dependencies) with the original input data, feeding this concatenated information into a feedforward neural network with ReLU activation for final prediction. This approach aims to enhance feature crossover capability and model fitting.
- Addressing Overfitting: The introduction of the time window concept is highlighted as a way to improve model flexibility and avoid overfitting issues common with long time-series training.
Datasets
The study utilized two primary sources of data:
- Official Monthly Public Health Scientific Data: Collected from the National Center for Disease Control and Prevention (CDC) in China, covering the period from 2012 to 2017.
- Daily Inpatient and Outpatient Medical Records: Obtained from Peking University Third Hospital, spanning from 2012 to 2020. This dataset was processed using Hadoop architecture, including HDFS, HBase, and Hive, to manage and analyze approximately 110,000 historical data points on infectious diseases.
Climate data was sourced from the national meteorological data department.
Key Insights and Ideas
- Beyond SEIR: The MSRD model demonstrates a significant improvement over traditional SEIR models, particularly in hospital settings, by overcoming limitations related to data acquisition and model complexity.
- The Power of Multi-Dimensional Data: Integrating diverse data sources (clinical, environmental, social) is crucial for accurate infectious disease prediction, as these factors are deeply interconnected with disease transmission.
- Hospital-Level Granularity Matters: While national data is valuable, fine-grained, daily data from individual institutions is essential for timely and actionable early warnings.
- Time Windows for Flexibility: The use of sliding time windows in the MSRD model is a key innovation that enhances the model's ability to adapt to varying data spans and avoid overfitting.
- Feature Importance: The study highlights the importance of carefully selected features, such as "student holiday" for HFMD, to capture specific disease transmission dynamics related to social activities.
Key Results
The MSRD model demonstrated superior performance compared to SEIR and several other machine learning and deep learning models (SVM, Lasso, Bayesian, DNN, LSTM, Bi-LSTM, GRU).
- MAE and RMSE: The paper reports an MAE of 0.6928 for hand, foot, and mouth disease (HFMD) and 1.3782 for influenza on test data. These values are significantly lower than those achieved by other models, indicating better prediction accuracy.
- Model Comparison: Table 1 provides a comprehensive comparison of MAE and RMSE across various models for different infectious diseases. In most cases, the MSRD model achieved the lowest MAE and competitive RMSE values.
- Prediction Accuracy: Experimental results show that the MSRD model can provide daily and instantaneous updates with accurate predictions for epidemic trends.
- Disease Trends: The study analyzed trends for seven common infectious diseases, noting that influenza and viral hepatitis showed slow growth, while HFMD, infectious diarrhea, scarlet fever, syphilis, and tuberculosis showed a year-on-year decrease, reflecting the effectiveness of public health interventions.
Limitations and Future Work
- Data Volume and Disease Rarity: The study acknowledges that the amount of data for some diseases is limited, and rare diseases (e.g., plague, cholera) are difficult to model due to their infrequent occurrence, making their trends incidental and unpredictable at the hospital level.
- Disease-Specific Characteristics: While the MSRD model incorporates climatic and social factors, it does not explicitly model the intrinsic characteristics of the infectious diseases themselves.
- Future Work Suggestions:
- Develop models for predicting various types of infectious diseases using real hospital medical record data combined with national data.
- Explore early warning and monitoring of emerging, unknown infectious diseases.
- Further investigate how to incorporate disease-specific characteristics into the MSRD model.
- Extend the model to all types of hospitals for broader infectious disease surveillance and management standardization.
- Address the issue of error accumulation in continuous future predictions by potentially reducing the influence of minor fluctuations in diagnosis numbers on model construction.
Summary updated