Semi-Supervised Multitask Learning on Multispectral Satellite Images Using Wasserstein Generative Adversarial Networks (GANs) for Predicting Poverty

Anthony Perez, Swetava Ganguli, Stefano Ermon, George Azzari, Marshall Burke, David B. LobellarXiv 2019 · arXiv

Introduction

Accurate poverty data is a cornerstone of effective policy-making and humanitarian aid. However, in many developing regions, particularly across Africa, traditional door-to-door surveys are prohibitively expensive and logistically difficult to conduct. Consequently, some nations may go over a decade without a representative poverty census. While high-resolution satellite imagery offers a scalable alternative, the vast majority of this data is unlabeled, lacking the corresponding economic ground truth necessary for supervised learning. This paper explores a semi-supervised multitask learning framework designed to bridge this data gap.

Methodology

The research proposes a deep learning pipeline that leverages vast amounts of unlabeled multispectral data through three primary strategies.

Semi-Supervised WGAN-GP

The core of the architecture is a Generative Adversarial Network (GAN) where the discriminator is adapted to perform classification. To address common GAN training instabilities, the authors utilize a Wasserstein GAN with Gradient Penalty (WGAN-GP). This approach allows the model to learn robust feature representations from the 95% of satellite imagery that lacks survey labels.

Multitask Learning and Regularization

To further regularize the model and prevent overfitting on sparse labels, the researchers employed multitask learning. The network was trained to simultaneously predict several geographic and human-activity indicators:

  • Nightlight intensity
  • Population density
  • Distance to the nearest road
  • Land cover classification

By forcing the model to identify features relevant to these diverse tasks, it learns more generalized geographic representations. Final poverty predictions for the Asset Wealth Index (AWI) were generated using a cross-validated Ridge Regression model applied to features extracted from the ResNet-50 based discriminator.

Data Sources

The study utilized approximately 22 TB of multispectral Landsat 7 imagery covering Africa from 2004 to 2015. This dataset includes nine spectral bands, providing information beyond the visible spectrum, such as thermal and infrared data. Ground-truth labels were sourced from Demographic and Health Surveys (DHS), though these labels were only available for a small fraction (approximately 5%) of the satellite images.

Spatial distribution of datasets across Africa: (a) unlabeled satellite imagery, (b) auxiliary task labels, and (c) sparse ground-truth survey labels.

Results and Discussion

The best-performing model achieved a Pearson correlation ($r^2$) of 0.57 for poverty prediction in Uganda (2011). This result significantly outperformed the baseline method of using raw nightlight luminosity alone ($r^2 = 0.484$).

A key finding was the impact of spectral bands on performance. Contrary to expectations, the model performed better using only RGB channels ($r^2 = 0.57$) than when using all nine multispectral bands ($r^2 = 0.53$). This suggests that the additional bands may have introduced noise that led to overfitting, despite the multitask regularization. Additionally, the "same init" strategy—initializing extra spectral channels with the mean of RGB weights—was found to be superior to random initialization.

Conclusion and Future Work

While the semi-supervised WGAN-GP framework demonstrates a viable path for poverty mapping in data-scarce regions, overfitting remains a significant challenge. The gap between training and validation performance indicates a need for further regularization. Future research will focus on expanding the multitask framework with additional auxiliary tasks and refining the GAN generator to produce higher-fidelity synthetic imagery, with the ultimate goal of producing high-granularity, real-time poverty maps for the entire continent.

Summary updated

Abstract

Obtaining reliable data describing local poverty metrics at a granularity that is informative to policy-makers requires expensive and logistically difficult surveys, particularly in the developing world. Not surprisingly, the poverty stricken regions are also the ones which have a high probability of being a war zone, have poor infrastructure and sometimes have governments that do not cooperate with internationally funded development efforts. We train a CNN on free and publicly available daytime satellite images of the African continent from Landsat 7 to build a model for predicting local economic livelihoods. Only 5% of the satellite images can be associated with labels (which are obtained from DHS Surveys) and thus a semi-supervised approach using a GAN (similar to the approach of Salimans, et al. (2016)), albeit with a more stable-to-train flavor of GANs called the Wasserstein GAN regularized with gradient penalty(Gulrajani, et al. (2017)) is used. The method of multitask learning is employed to regularize the network and also create an end-to-end model for the prediction of multiple poverty metrics.
BibTeX
@misc{perez2019semi,
  author = {Anthony Perez and Swetava Ganguli and Stefano Ermon and George Azzari and Marshall Burke and David B. Lobell},
  title = {{Semi-Supervised Multitask Learning on Multispectral Satellite Images Using Wasserstein Generative Adversarial Networks (GANs) for Predicting Poverty}},
  year = {2019},
  eprint = {1902.11110},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1902.11110},
}
Figure