Inadequate methods undermine a study of malaria, deforestation and trade

Nikolas KuschnigNature Communications 2021 · DOI · OpenAlex

Abstract

In a recent study, Chaves et al. find international consumption and trade to be major drivers of ‘malaria risk’ via deforestation. Their analysis is based on a counterfactual ‘malaria risk’ footprint, defined as the number of malaria cases in absence of two malaria interventions, which is constructed using linear regression. In this letter, I argue that their study hinges on an obscured weighting scheme and suffers from methodological flaws, such as disregard for sources of bias. When addressed properly, these issues nullify results, overturning the significance and reversing the direction of the claimed relationship.

Introduction: The Spurious Regression Trap

A high-profile 2020 study by Chaves et al. claimed to establish a causal link between global trade-driven deforestation and increased malaria incidence. However, a rigorous re-evaluation by Nikolas Kuschnig, published as a "Matters Arising" article in Nature Communications, suggests these findings are the result of statistical artifacts rather than ecological signals.

The core issue lies in "spurious regression," a common pitfall in environmental economics. When two variables—such as cumulative tree loss and malaria interventions—both trend steadily over time, a simple linear model may report a high correlation ($R^2$) even in the absence of a causal relationship. This is often referred to as the "storks deliver babies" statistical trap.

Malaria cases aggregated globally from 2000 to 2015, showing a peak around 2005 followed by a decline.

Methodological Flaws and Hidden Weights

Kuschnig’s forensic analysis of the original replication files revealed several critical oversights that undermined the study's validity:

  • Obscured Weighting: The original study employed a weighted regression that was not disclosed in the methodology. These weights were only discoverable by unhiding rows in the authors' provided spreadsheets.
  • Inflated Significance: By duplicating observations to create these weights, the original authors artificially inflated their sample size, leading to underestimated standard errors and exaggerated statistical significance.
  • Non-Stationarity: Testing revealed that the variables involved are non-stationary, meaning their statistical properties change over time. This makes standard linear regression inappropriate without specific corrections.
Cumulative tree cover loss in tropical countries showing a steady upward trend from 2000 to 2015.

Re-evaluating the Evidence

When standard econometric corrections are applied, the original conclusions dissolve. Kuschnig tested four alternative model specifications, finding that the significance of tree loss disappeared in unweighted models. Most strikingly, when accounting for general temporal trends, the coefficient for tree loss actually switched signs, suggesting an implausible negative relationship—a clear indicator of model mis-specification.

Furthermore, the study suffered from "simultaneity bias." Because governments often increase malaria interventions (like bed nets or ACT treatments) in response to rising case numbers, regressing incidence on interventions violates the assumption of independent variables.

Conclusion and Future Directions

While Kuschnig does not dismiss the possibility that deforestation impacts malaria, he demonstrates that the aggregated global approach used by Chaves et al. is insufficient to prove it. To advance this field, future research should utilize subnational data to account for regional heterogeneity and employ advanced causal inference methods, such as instrumental variables, to manage reciprocal causation between disease and intervention. Finally, methodological transparency regarding weighting and data manipulation is essential for peer validation.

Summary updated

Abstract

In a recent study, Chaves et al. find international consumption and trade to be major drivers of ‘malaria risk’ via deforestation. Their analysis is based on a counterfactual ‘malaria risk’ footprint, defined as the number of malaria cases in absence of two malaria interventions, which is constructed using linear regression. In this letter, I argue that their study hinges on an obscured weighting scheme and suffers from methodological flaws, such as disregard for sources of bias. When addressed properly, these issues nullify results, overturning the significance and reversing the direction of the claimed relationship. Nonetheless, I see great potential in the mixed methods approach and conclude with recommendations for future studies.
BibTeX
@inproceedings{kuschnig2021inadequate,
  author = {Nikolas Kuschnig},
  title = {{Inadequate methods undermine a study of malaria, deforestation and trade}},
  booktitle = {Nature Communications},
  year = {2021},
  doi = {10.1038/s41467-021-22514-4},
}
Figure