Missing in space: an evaluation of imputation methods for missing data in spatial analysis of risk factors for type II diabetes

Jannah Baker, Nicole White, Kerrie MengersenInternational Journal of Health Geographics 2014 · DOI

TL;DR

This paper tackles the common problem of missing data in spatial health analyses, specifically for type II diabetes risk factors in Queensland, Australia. They evaluated three imputation methods and found that simple mean imputation was the most accurate for their dataset, highlighting that the most complex method isn't always the best. This work is crucial for improving the reliability of spatial health studies and informing public health policy.

What Problem Does This Paper Address?

Spatial analysis is a powerful tool for identifying geographic risk factors for diseases, which can then inform public health interventions and resource allocation. However, a significant challenge in this field is the prevalence of incomplete spatial health survey data. This missingness can range from a few data points to substantial gaps across many variables and regions. When analyzing such data, the choice of how to handle missing values (imputation) is critical, as it can significantly impact the accuracy and validity of the inferences drawn. Without appropriate imputation, spatial analyses might produce biased results or be unable to utilize valuable data, hindering effective public health decision-making. This paper specifically addresses this by evaluating different imputation methods for spatial analysis of type II diabetes (DM II) risk factors in Queensland, Australia.

Methods

The core of this paper lies in its evaluation of three distinct imputation methods for spatial health survey data:

  • Mean Imputation: This is the simplest method, where missing values for a particular covariate are replaced with the average of the observed values for that same covariate.
  • Multivariate Normal (MVN) Imputation: This method leverages the correlations between different covariates. It estimates a variance-covariance matrix for the covariates and uses this to predict missing values, assuming the data follows a multivariate normal distribution.
  • Conditional Autoregressive (CAR) Prior Imputation: This approach accounts for spatial correlation. It estimates the spatial relationships between neighboring regions for each covariate and uses these spatial patterns to impute missing values.

To select the most appropriate method, the researchers employed a cross-validation approach. They split their data into training and testing sets, artificially introduced missingness into the test set, and then used each imputation method on the training set to predict the missing values in the test set. The accuracy of these predictions was then evaluated using Root Mean Squared Error (RMSE) and bias metrics.

Datasets

The study utilized three primary datasets for its analysis:

  • National Diabetes Services Scheme (NDSS) database (2011): This provided diabetic notification data, used to estimate DM II prevalence. It also contained data on socioeconomic status (SES) and the proportion of the population over 45 years old.
  • 2011 Census data from the Australian Bureau of Statistics (ABS): This provided estimated resident population (ERP) for each Local Government Area (LGA).
  • Queensland self-reported health status survey (2009-2010): This survey provided estimates of key health indicators, including Body Mass Index (BMI), proportion of daily smokers, physical activity levels, and fruit and vegetable intake, weighted by age and gender.

The analysis focused on 71 Local Government Areas (LGAs) in Queensland, Australia.

Key Insights and Ideas

  • Complexity is not always better: The study's most striking finding is that the simplest imputation method, mean imputation, outperformed more complex methods (MVN and CAR priors) in terms of accuracy for their specific dataset. This challenges the assumption that more sophisticated methods are always superior.
  • Dataset characteristics matter: The paper suggests that the pattern of missingness and the correlations within the data heavily influence which imputation method is most suitable. In their case, the missingness was not random and was related to population size, which might have favored mean imputation.
  • Spatial correlation doesn't guarantee superiority: While CAR imputation explicitly models spatial correlation, it did not yield better results than mean imputation in this specific study. This implies that the benefits of spatial modeling might be diminished when other factors, like the extent of missingness, play a more dominant role.
  • The importance of validation: The use of a rigorous cross-validation approach was crucial for objectively comparing the imputation methods and selecting the most accurate one for their specific application.

Key Results

  • Mean imputation was the most accurate: Mean imputation achieved the lowest overall Root Mean Squared Error (RMSE) of 32.5. In contrast, CAR prior imputation had an RMSE of 46.1, and MVN imputation had the highest RMSE (71.1 for Poisson and 72.7 for Binomial models).
  • Bias was minimal with mean imputation: Mean imputation resulted in zero bias by definition. While MVN and CAR imputation also showed relatively low bias, MVN imputation produced greater uncertainty in bias (larger credible intervals).
  • Significant geographic variation in DM II risk: The study found a six-fold difference in DM II relative risk (RR) across Queensland LGAs, ranging from 0.48 to 3.07.
  • Socioeconomic status is a key predictor: Socioeconomic status (SES) was identified as the only variable significantly associated with DM II risk across both Poisson and Binomial models. Higher SES was associated with a decreased risk of DM II.
  • Other lifestyle factors were not significant at the regional level: Contrary to some individual-level studies, factors like BMI, smoking, physical activity, and fruit/vegetable intake were not found to be significantly associated with DM II risk at the regional level in this dataset.

Limitations and Future Work

  • Missing data not at random: The authors acknowledge that the missing data in their dataset was not missing at random, which could influence the performance of imputation methods. Multivariate normal imputation might be more suitable for datasets with missingness at random and high covariate correlations.
  • Small test datasets in cross-validation: The cross-validation approach used a relatively small proportion of data for testing, which the authors note should lead to caution in interpreting the imputation results.
  • Diabetic counts based on notification data: The use of notification data means potential measurement bias is present, and the accuracy of these counts is unknown.
  • Self-reported lifestyle data: Lifestyle data was based on self-report, which is subject to inaccuracies.
  • Future work: The authors suggest that their methodology is applicable to spatial analyses of diabetes in other regions, as well as other diseases. They also highlight the potential for using other information, such as spatial correlation and covariate correlations, in imputation methods, especially in cases with different missing data patterns.

Summary updated

Abstract

Background: Spatial analysis is increasingly important for identifying modifiable geographic risk factors for disease. However, spatial health data from surveys are often incomplete, ranging from missing data for only a few variables, to missing data for many variables. For spatial analyses of health outcomes, selection of an appropriate imputation method is critical in order to produce the most accurate inferences. Methods: We present a cross-validation approach to select between three imputation methods for health survey data with correlated lifestyle covariates, using as a case study, type II diabetes mellitus (DM II) risk across 71 Queensland Local Government Areas (LGAs). We compare the accuracy of mean imputation to imputation using multivariate normal and conditional autoregressive prior distributions. Results: Choice of imputation method depends upon the application and is not necessarily the most complex method. Mean imputation was selected as the most accurate method in this application. Conclusions: Selecting an appropriate imputation method for health survey data, after accounting for spatial correlation and correlation between covariates, allows more complete analysis of geographic risk factors for disease with more confidence in the results to inform public policy decision-making.
BibTeX
@article{baker2014missing,
  author = {Jannah Baker and Nicole White and Kerrie Mengersen},
  title = {{Missing in space: an evaluation of imputation methods for missing data in spatial analysis of risk factors for type II diabetes}},
  journal = {International Journal of Health Geographics},
  year = {2014},
  doi = {10.1186/1476-072x-13-47},
}
Figure