Introduction
Monitoring progress toward Sustainable Development Goals (SDGs) requires frequent and granular data on local livelihood indicators, such as poverty levels, population density, and health outcomes. Traditionally, these metrics are derived from nationally representative household surveys, such as the Demographic and Health Surveys (DHS). However, these surveys are resource-intensive, infrequent, and often fail to capture data from remote or marginalized populations.
While satellite imagery has emerged as a scalable alternative, it often lacks the "human-scale" resolution necessary to identify specific indicators of development, such as road quality, vehicle types, or local infrastructure. This research introduces a machine learning framework that leverages crowd-sourced, street-level imagery from Mapillary to predict these critical livelihood indicators in developing regions like India and Kenya.
Methodology
The researchers developed two primary approaches to extract socioeconomic signals from unstructured, crowd-sourced photography:
1. Object-Based Clustering (Obj-ClusterWise)
Using a panoptic segmentation model trained on the Mapillary Vistas dataset, the framework identifies and counts 65 distinct object classes—ranging from street lights and manholes to bicycles and bridges—within geographic clusters. These counts serve as interpretable features for downstream predictive models like Random Forests or Multi-Layer Perceptrons (MLPs).
2. Graph Convolutional Networks (GCN)
To capture the spatial relationships within a neighborhood, the team modeled geographic clusters as graphs. In this architecture, individual images serve as nodes containing visual features, while edges represent spatial proximity. This allows the model to learn the structural context of a community—for instance, the proximity of paved roads to commercial signage—rather than treating images as isolated data points.
Data and Ground Truth
The study utilized approximately 1.1 million images from India and 156,000 from Kenya, sourced via the Mapillary API. These images were spatially matched to ground truth labels from:
- Wealth and Health (BMI): 2015-16 DHS for India and 2014 DHS for Kenya.
- Population Density: Facebook’s High Resolution Population Density Maps.
Key Findings
The GCN-based approach consistently outperformed baseline methods, demonstrating the value of spatial context in socioeconomic prediction.
- High Predictive Accuracy: The model achieved 81.06% accuracy for poverty prediction in India and 77.00% in Kenya. Notably, it predicted population density in India with 94.71% accuracy.
- Health Indicator Feasibility: The framework predicted Women's BMI in India with 89.56% accuracy, a significant result given that health indicators are typically difficult to estimate from visual data.
- Interpretability: Object-level analysis revealed that features like "street lights" and "manholes" are strong positive proxies for development, while "nature-terrain" or "exposed ground" often correlate with lower wealth.
- Universal Markers: Models trained on pooled data from both countries maintained high performance, suggesting that visual indicators of development are relatively consistent across different geographic contexts.
Limitations and Future Outlook
Despite its success, the approach faces challenges related to "road bias," as Mapillary data is primarily collected along navigable paths. Furthermore, temporal gaps between image uploads and official surveys can introduce noise. Future research may expand to other indicators, such as child mortality or access to clean water, and investigate methods to mitigate the inherent biases in crowd-sourced data collection.
Summary updated