Methodological Guidelines to Estimate Population-based Health Indicators Using Linked Data and/or Machine Learning Techniques

Romana Haneef, Mariken Tijhuis, Rodolphe Thiébaut, Ondřej Májek, Ivan Pristaš, Hanna Tolenan, Anne Gallay

Introduction

The integration of administrative data and artificial intelligence offers unprecedented opportunities for public health monitoring. By linking hospital records, mortality registries, and insurance claims, researchers can generate a more holistic view of population health than is possible with single-source datasets. However, the use of linked administrative data is fraught with challenges, including data variability, lack of interoperability, and a shortage of specialized data science expertise. This paper presents the first systematic methodological guidelines designed to harmonize the estimation of health indicators using linked data and machine learning (ML) techniques across European countries.

Methodology

The guidelines were developed through a rigorous three-step systematic approach:

  1. Literature Review: A comprehensive screening of 333 citations from PubMed led to the selection of 118 peer-reviewed methodological articles and systematic reviews.
  2. Case Study Identification: The authors identified 16 "inspiring examples" from ten European countries that successfully implemented linked data or ML strategies to inform public health actions.
  3. Checklist Synthesis: Best practices from these 134 sources were synthesized into a 9-item methodological checklist.

The framework addresses the entire research lifecycle, utilizing the PICO (Population, Intervention, Comparator, Outcome) criteria for research questions and incorporating advanced ML workflows such as the ReliefF algorithm for feature selection.

The 9-Point Methodological Checklist

The core contribution of this study is a structured checklist intended to guide researchers through the complexities of big data in public health:

  1. Rationale and Objective: Defining the core research question.
  2. Study Design: Selecting appropriate epidemiological designs (e.g., cross-sectional, cohort, or case-control).
  3. Linked Data Sources: Identifying and integrating relevant administrative and clinical databases.
  4. Population and Sample Size: Establishing inclusion/exclusion criteria and ensuring statistical power.
  5. Study Outcomes: Defining health indicators at various geographical and demographic levels.
  6. Data Preparation: Managing raw data extraction and preprocessing for ML.
  7. Data Analysis: Applying statistical or ML models and conducting variable selection.
  8. Sensitivity Analysis: Evaluating the robustness of findings against alternative assumptions.
  9. Study Limitations: Transparently reporting constraints related to data privacy, quality, or misclassification.

Technical Considerations for Real-World Data

The guidelines bridge classical statistics (e.g., Cox regression) and modern ML (e.g., Random Forest, Neural Networks), offering specific strategies for common data challenges:

  • Imbalanced Datasets: Recommending down-sampling or over-sampling to prevent model bias toward majority classes.
  • Missing Data: Categorizing missingness (MCAR, MAR, MNAR) and suggesting advanced imputation methods such as MICE (Multivariate Imputation by Chained Equation) or deep learning tools like DataWig.
  • Validation: Emphasizing the necessity of k-fold cross-validation and Area Under the Curve (AUC) metrics to ensure model generalizability.

Conclusion

These guidelines provide a foundational framework for improving population health information systems. By standardizing the use of linked data and ML, researchers can produce high-quality evidence to support public health policy. Future efforts should focus on enhancing "knowledge translation" for policymakers and harmonizing GDPR interpretations to facilitate cross-border health research within the EU.

Summary updated

Abstract

Abstract Background: The capacity to use data linkage and artificial intelligence to estimate and predict health indicators varies across European countries. However, the estimation of health indicators from linked administrative data is challenging due to several reasons such as variability in data sources and data collection methods resulting in reduced interoperability at various levels and timeliness, availability of a large number of variables, lack of skills and capacity to link and analyze big data. The main objective of this study is to develop the methodological guidelines calculating population-based health indicators to guide European countries using linked data and/or machine learning (ML) techniques with new methods. Method: We have performed the following step-wise approach systematically to develop the methodological guidelines: i. Scientific literature review, ii. Identification of inspiring examples from European countries, and iii. Developing the checklist of guidelines contents. Results: We have developed the methodological guidelines, which provide a systematic approach for studies using linked data and/or ML-techniques to produce population-based health indicators. These guidelines include a detailed checklist of the following items: rationale and objective of the study (i.e., research question), study design, linked data sources, study population/sample size, study outcomes, data preparation, data analysis (i.e., statistical techniques, sensitivity analysis and potential issues during data analysis) and study limitations. Conclusions: This is the first study to develop the methodological guidelines for studies focused on population health using linked data and/or machine learning techniques. These guidelines would support researchers to adopt and develop a systematic approach for high-quality research methods. There is a need for high-quality research methodologies using more linked data and ML-techniques to develop a structured cross-disciplinary approach for improving the population health information and thereby the population health.
BibTeX
@inproceedings{haneef2022methodological,
  author = {Romana Haneef and Mariken Tijhuis and Rodolphe Thiébaut and Ondřej Májek and Ivan Pristaš and Hanna Tolenan and Anne Gallay},
  title = {{Methodological Guidelines to Estimate Population-based Health Indicators Using Linked Data and/or Machine Learning Techniques}},
  booktitle = {Archives},
  year = {2022},
  doi = {10.1186/s13690-021-00770-6},
}
Figure