Alternative approaches for studying humanitarian interventions: propensity score methods to evaluate reintegration packages impact on depression, PTSD, and function impairment among child soldiers in Nepal

Brandon A. Kohrt, Matthew D. Burkey, Elizabeth A. Stuart, S KoiralaCambridge 2015 · DOI

Abstract

In a humanitarian crisis in which a non-randomized intervention assignment protocol was employed, the statistical technique of PSMs addressed differences in covariate distribution between child soldiers who received different integration packages. Our analysis did not demonstrate significant changes in psychosocial outcomes for ERPs and ORPs. We suggest the use of PSMs in evaluating non-randomized interventions in humanitarian crises when non-randomized conditions are not utilized.

Introduction

Evaluating Mental Health and Psychosocial Support (MHPS) programs in humanitarian crises presents significant ethical and logistical challenges. While Randomized Controlled Trials (RCTs) are the gold standard for establishing causality, they are often indefensible in post-conflict or disaster settings where withholding aid for a control group is problematic. This study explores the application of Propensity Score Methods (PSMs) as a robust alternative to evaluate the impact of reintegration packages on former child soldiers in Nepal.

Methodology

The study followed a cohort of 258 former child soldiers after the People’s War (1996–2006). Participants selected one of three paths: Education Reintegration Packages (ERP; 54.7%), Other Reintegration Packages (ORP, such as vocational training; 22.9%), or no formal package.

Because the children chose their own paths, the groups were fundamentally different at baseline (selection bias). For instance, younger children were more likely to choose education, while those from marginalized castes faced different barriers. To address this, the researchers employed PSMs using the twang package in R. They calculated multinomial propensity scores based on 13 baseline variables, including age, sex, caste, trauma history, and initial symptom scores. These scores were used to weight participants, effectively "balancing" the groups so they became statistically comparable, mimicking a randomized environment. The primary outcomes were measured using the Depression Self-Rating Scale (DSRS), the Child PTSD Symptom Scale (CPSS), and the Child Functional Impairment (CFI) scale.

Results

The PSM approach successfully improved group balance. The Maximum Absolute Standardized Mean Difference (ASMD)—a measure of group imbalance—dropped from over 0.50 to below 0.20 for all measured variables after weighting.

However, the weighted analysis did not reveal statistically significant improvements in mental health outcomes over the one-year follow-up period:

  • Depression (DSRS): ERP and ORP showed non-significant reductions of 0.59 and 0.60 points, respectively (95% CI −1.97 to 0.70 for ERP).
  • PTSD (CPSS): Both intervention groups showed slight, non-significant increases in symptoms (1.15 for ERP; 0.66 for ORP).
  • Functional Impairment (CFI): Non-significant increases were also observed (0.91 for ERP; 1.05 for ORP).

Discussion

The lack of significant findings suggests that the specific reintegration packages provided may not have been sufficient to address complex psychological trauma within a single year. Additionally, the authors suggest that because all children in these districts had access to some level of community-based psychosocial support, the specific impact of the reintegration packages may have been diluted. Other factors, such as the relatively small sample size (n=258) and unobserved confounders like parental support, may have also influenced the results.

Despite the "null" results regarding the interventions themselves, the study validates the utility of PSMs in humanitarian research. By accounting for observed confounders, PSMs allow researchers to derive more accurate insights from non-randomized "natural experiments" in messy, real-world data.

Conclusion

This research demonstrates that PSMs are a viable tool for evaluating humanitarian interventions when RCTs are not feasible. While the specific packages studied did not show immediate psychological benefits, the methodology provides a rigorous framework for future impact assessments. The authors call for PSMs to be planned at the design stage of humanitarian programs to ensure the collection of comprehensive baseline data, enabling more robust evaluations in the future.

Summary updated

Abstract

In a humanitarian crisis in which a non-randomized intervention assignment protocol was employed, the statistical technique of PSMs addressed differences in covariate distribution between child soldiers who received different integration packages. Our analysis did not demonstrate significant changes in psychosocial outcomes for ERPs and ORPs. We suggest the use of PSMs in evaluating non-randomized interventions in humanitarian crises when non-randomized conditions are not utilized.
BibTeX
@inproceedings{kohrt2015alternative,
  author = {Brandon A. Kohrt and Matthew D. Burkey and Elizabeth A. Stuart and S Koirala},
  title = {{Alternative approaches for studying humanitarian interventions: propensity score methods to evaluate reintegration packages impact on depression, PTSD, and function impairment among child soldiers in Nepal}},
  booktitle = {Cambridge},
  year = {2015},
  doi = {10.1017/gmh.2015.13},
}
Figure