Multi-Task Deep Learning for Predicting Poverty From Satellite Images

Shailesh M. Pandey, Tushar Agarwal, Narayanan C. KrishnanAAAI 2018 · DOI

Abstract

Traditional poverty tracking in developing nations is often hindered by the high cost and low frequency of ground-level surveys. This paper presents a two-step deep learning framework designed to estimate poverty levels in rural India using high-resolution daytime satellite imagery. By training a multi-task fully convolutional network to identify physical developmental proxies—such as roofing materials, lighting sources, and water access—the system can accurately predict household income distributions. The model achieves a 96.9% accuracy in binary poverty classification, demonstrating that physical infrastructure is a more granular and reliable indicator of economic status than traditional nighttime luminosity data.

Introduction

In many developing regions, economic data is sparse. Governments often rely on decennial censuses or infrequent surveys, leaving policy-makers with outdated information. While satellite imagery offers a low-cost, high-frequency alternative, extracting economic signals from pixels is challenging. Previous efforts primarily utilized nighttime lights as a proxy; however, this method fails in the world's poorest rural areas where electricity is non-existent. This research addresses this gap by identifying daytime visual proxies—such as roof types and proximity to water sources—that correlate strongly with poverty levels.

Methodology

The researchers implemented a two-stage "proxy-based" deep learning architecture:

  1. Multi-Task Fully Convolutional Network (FCN): A deep learning model was developed to simultaneously predict three key developmental indicators: roof material, source of lighting, and source of drinking water. The architecture utilizes a shared convolutional "trunk" to learn general features like edges and textures, which then branches into task-specific layers. This multi-task approach not only improved efficiency, reducing training time by 66%, but also acted as a form of regularization, forcing the model to learn more robust environmental features.
Close-up of a dense rural settlement (A1), illustrating the complex visual patterns used to identify roofing materials and housing density.
  1. Income Estimation Model: The predicted distributions of these physical proxies are fed into a four-layer fully connected network. This model estimates the proportion of households within a region falling into three specific monthly income brackets: below ₹5,000, ₹5,000–₹10,000, and above ₹10,000.

Datasets

The study focused on Uttar Pradesh, India’s most populous state. The dataset comprised 47,120 villages, filtered for those with at least 100 households. Ground truth data was sourced from the 2011 Census of India and the Socio-Economic Caste Census. High-resolution imagery (zoom level 16) was obtained via the Google Static Maps API to provide the visual input for the FCN.

Examples of satellite imagery across different village types and infrastructure features (A1-C2) in the Uttar Pradesh dataset.

Key Results and Insights

  • Performance Parity: The model trained on predicted satellite features performed nearly as well as a model trained on actual ground-truth census data, indicating high accuracy in the computer vision component.
  • Feature Extraction: Visualizing filter responses revealed that the network automatically learned to identify roads, water bodies, and agricultural land. Task-specific branches showed specialized focus: the lighting branch prioritized roads, while the roofing branch focused strictly on human settlements.
  • Classification Accuracy: For binary "poor" vs. "not poor" classification, the model reached an accuracy of 96.9%, significantly outperforming baseline methods.

Discussion and Limitations

While highly effective, the model faces challenges regarding spatial mismatch, as income data was only available at the sub-district level while proxies were village-level. Additionally, because the model was trained on rural India, its visual proxies may not immediately generalize to other regions like Sub-Saharan Africa without further transfer learning. Future work aims to tighten the temporal alignment between census data and satellite imagery to further refine prediction accuracy.

Summary updated

BibTeX
@inproceedings{pandey2018multi,
  author = {Shailesh M. Pandey and Tushar Agarwal and Narayanan C. Krishnan},
  title = {{Multi-Task Deep Learning for Predicting Poverty From Satellite Images}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2018},
  doi = {10.1609/aaai.v32i1.11416},
}
Figure