Development and validation of a risk prediction model for diabetic retinopathy in type 2 diabetic patients

Chengjun Zhu, Jiaxi Zhu, Lei Wang, Shizheng Xiong, Yijian Zou, Jing Huang, Huimin Xie, Wenye Zhang, Huiqun Wu, Yun LiuScientific 2023 · DOI

TL;DR

This paper presents a novel risk prediction model for diabetic retinopathy (DR) in type 2 diabetes patients, developed using logistic regression on data from meta-analyses of known risk factors. The model demonstrates high predictive accuracy (AUC of 0.912) and has been validated externally, offering a promising tool for individualized risk assessment and early intervention.

What Problem Does This Paper Address?

Diabetic retinopathy (DR) is a leading cause of blindness globally, stemming from chronic complications of diabetes mellitus (DM). Type 2 DM (T2DM) is a growing epidemic, and its complications, including DR, place a significant burden on healthcare systems and reduce patients' quality of life. While the importance of early screening and intervention for DR is recognized, there's a need for more personalized risk assessment tools to identify individuals at higher risk and guide targeted interventions. Existing predictive models often rely on observational data, which can limit their quality, and there's a call for models that are interpretable and easily applicable by both clinicians and patients.

Methods

The researchers employed a multi-stage approach to develop and validate their DR risk prediction model.

  1. Systematic Review and Meta-Analysis: They conducted a comprehensive search of multiple databases to identify relevant meta-analyses on DR risk factors in T2DM patients. Inclusion and exclusion criteria were applied to select studies that provided pooled odds ratios (OR) or relative risks (RR) for various risk factors.
  2. Logistic Regression (LR) Modeling: The extracted pooled OR/RR values were used to calculate β coefficients for each risk factor. These coefficients were then incorporated into a logistic regression model to construct the DR risk prediction model. The model accounts for 12 identified risk factors, including demographic, lifestyle, and clinical variables.
  3. External Validation: To assess the model's real-world performance, an electronic patient-reported outcome questionnaire was developed. This questionnaire was administered to 60 T2DM patients (30 with DR and 30 without) to collect data for an external validation.
  4. Performance Evaluation: The model's predictive accuracy was evaluated using Receiver Operating Characteristic (ROC) curves and calculating the Area Under the Curve (AUC), sensitivity, and specificity.

Datasets

The primary data for model construction was derived from eight meta-analyses that collectively included 15,654 cases. These meta-analyses provided pooled odds ratios (OR) or relative risks (RR) for 12 identified risk factors associated with DR in T2DM patients.

For external validation, a newly collected dataset of 60 T2DM patients (30 with DR, 30 without) was used. This data was gathered through an electronic patient-reported outcome questionnaire.

Key Insights and Ideas

  • Interpretable Model for Personalized Risk: The study successfully developed an interpretable logistic regression model, which is crucial for both clinicians and patients to understand the factors contributing to DR risk and to make informed decisions.
  • Leveraging Meta-Analyses for Robustness: By synthesizing data from multiple meta-analyses, the researchers aimed to overcome the limitations of individual observational studies and build a more robust model.
  • Comprehensive Risk Factor Identification: The model incorporates a wide range of risk factors, including demographic (gender, residence), lifestyle (smoking), clinical (hypertension, HbA1c, fasting plasma glucose, diabetes duration, insulin treatment, intensive glycemic control), and specific conditions (myopia, bariatric surgery, lipid-lowering drug use).
  • High Predictive Power: The model achieved a strong AUC of 0.912 in external validation, indicating a high level of accuracy in distinguishing between patients at high and low risk for DR.
  • Practical Application Potential: The development of a prototype Android application demonstrates the authors' intent to translate this model into a user-friendly tool for practical clinical use and patient self-management.

Key Results

  • The developed logistic regression model identified 12 significant risk factors for DR in T2DM patients.
  • The model achieved an Area Under the ROC Curve (AUC) of 0.912 during external validation.
  • The model demonstrated a sensitivity of 0.867 and a specificity of 0.867.
  • The study provides specific β coefficients for each risk factor, allowing for the calculation of an individual's DR risk score. For example, bariatric surgery had a β coefficient of -0.942, indicating a protective effect.
  • An example case demonstrated that a 60-year-old male diabetic patient with specific risk factors had a predicted risk score 2.75 times higher than the average DR prevalence in a local diabetic population.

Limitations and Future Work

  • Heterogeneity in Meta-Analyses: The authors acknowledge that the systematic review and meta-analysis process can be subject to heterogeneity due to variations in research design, methods, and cohort characteristics. They suggest further investigation and subgroup analysis to mitigate this.
  • Incomplete Risk Factor Data: The risk factors included in the original literature might not have been exhaustive, potentially affecting the comprehensiveness and reliability of the model.
  • Potential for Bias: The research design, data collection, and statistical processing may involve man-made subjective factors that could introduce deviations.
  • Generalizability: DR prevalence can vary across different populations, suggesting that the model's implementation might need tailoring to specific demographic groups to optimize performance.
  • Future Work: The authors emphasize the need for further verification of the model with large-scale applications to confirm its representativeness and accuracy. They also suggest considering the relationship between factors and model performance in future research.

Summary updated

Abstract

To establish a risk prediction model and make individualized assessment for the susceptible diabetic retinopathy (DR) population in type 2 diabetic mellitus (T2DM) patients. According to the retrieval strategy, inclusion and exclusion criteria, the relevant meta-analyses on DR risk factors were searched and evaluated. The pooled odds ratio (OR) or relative risk (RR) of each risk factor was obtained and calculated for β coefficients using logistic regression (LR) model. Besides, an electronic patient-reported outcome questionnaire was developed and 60 cases of DR and non-DR T2DM patients were investigated to validate the developed model. Receiver operating characteristic curve (ROC) was drawn to verify the prediction accuracy of the model. After retrieving, eight meta-analyses with a total of 15,654 cases and 12 risk factors associated with the onset of DR in T2DM, including weight loss surgery, myopia, lipid-lowing drugs, intensive glucose control, course of T2DM, glycated hemoglobin (HbA1c), fasting plasma glucose, hypertension, gender, insulin treatment, residence, and smoking were included for LR modeling. These factors, followed by the respective β coefficient was bariatric surgery (- 0.942), myopia (- 0.357), lipid-lowering drug follow-up < 3y (- 0.994), lipid-lowering drug follow-up > 3y (- 0.223), course of T2DM (0.174), HbA1c (0.372), fasting plasma glucose (0.223), insulin therapy (0.688), rural residence (0.199), smoking (- 0.083), hypertension (0.405), male (0.548), intensive glycemic control (- 0.400) with constant term α (- 0.949) in the constructed model. The area under receiver operating characteristic curve (AUC) of the model in the external validation was 0.912. An application was presented as an example of use. In conclusion, the risk prediction model of DR is developed, which makes individualized assessment for the susceptible DR population feasible and needs to be further verified with large sample size application.
BibTeX
@inproceedings{zhu2023development,
  author = {Chengjun Zhu and Jiaxi Zhu and Lei Wang and Shizheng Xiong and Yijian Zou and Jing Huang and Huimin Xie and Wenye Zhang and Huiqun Wu and Yun Liu},
  title = {{Development and validation of a risk prediction model for diabetic retinopathy in type 2 diabetic patients}},
  booktitle = {Scientific},
  year = {2023},
  doi = {10.1038/s41598-023-31463-5},
}
Figure