Multi-View Learning from Disparate Sources for Poverty Mapping

Neeti PokhriyalAAAI 2019 · DOI

Abstract

Accurate and timely poverty mapping is critical for effective policy intervention, yet traditional census-based methods are often infrequent and costly. This paper proposes a multi-view learning framework that fuses disparate data sources—specifically mobile phone records and environmental data (satellite imagery, weather, and maps)—to generate high-resolution poverty estimates. By employing Gaussian Process Regression (GPR) with uncertainty-weighted fusion and Discriminative Factorized Subspaces (DFS), the model achieves a Pearson correlation of 0.91 with ground-truth data in Senegal. The framework is designed to be robust to missing data sources and provides interpretable results for decision-makers.

Introduction

Traditional socioeconomic monitoring relies on intensive household surveys and census data, which are often updated only once a decade. This "statistical tragedy" leaves policy-makers with outdated information. While modern "big data" sources like mobile phone logs and satellite imagery offer a more frequent alternative, they are inherently heterogeneous, varying in spatial granularity, noise levels, and completeness.

This research addresses the challenge of integrating these disparate data "views." Existing machine learning models often struggle with the messy reality of real-world data, either assuming total independence or total dependence between sources. The proposed framework seeks to bridge this gap by learning factorized representations that separate shared information from source-specific noise.

Methodology

The framework utilizes two primary methodologies to handle regression and classification tasks:

Gaussian Process Regression (GPR) for Fusion

To predict continuous poverty values, independent GPR models are trained on two distinct views:

  1. Mobile View: Features derived from aggregated Call Detail Records (CDR).
  2. Environmental View: Data from satellite imagery, ground-based weather measurements, and Open Street Maps (OSM).

The GPR models utilize specialized spatial kernels that incorporate both feature data and physical coordinates (latitude/longitude). Instead of simple averaging, the final prediction is an uncertainty-weighted combination. Predictions from a source with higher predictive variance (uncertainty) are given less weight, ensuring a more robust final estimate.

Discriminative Factorized Subspace (DFS)

For classification tasks (e.g., identifying "poor" vs. "not poor" regions), the paper introduces the Discriminative Factorized Subspace (DFS) method. This approach decomposes data into:

  • Shared Subspaces: Capturing common variance across all data sources.
  • Private Subspaces: Capturing variance unique to a specific source.

This factorization is crucial for robustness; if one data source (like mobile data) is missing for a specific region, the model can still leverage the shared latent space to make an informed prediction.

Data and Experimental Setup

The model was evaluated using data from Senegal at the commune level. The target labels were Multidimensional Poverty Index (MPI) values derived from official census data. The input features included aggregated CDRs for the mobile view and a combination of satellite, weather, and OSM data for the environmental view.

Results and Discussion

The multi-view GPR model demonstrated significant predictive power, achieving a Pearson correlation of 0.91 with ground-truth census data. This performance exceeded that of single-source models, which achieved correlations of 0.89 (mobile only) and 0.84 (environmental only).

Predicted vs. Actual Multidimensional Poverty Index (MPI) at the commune level. The plot shows a strong correlation between predicted and actual values, with distinct clusters for urban and rural areas.

As shown in the figure above, the model effectively captures the variance in MPI across both urban and rural communes. The size of the points indicates the predictive variance, highlighting the model's ability to quantify its own uncertainty. The concentration of points along the diagonal line confirms the high correlation reported.

In classification tasks, the DFS approach achieved 83% accuracy. Furthermore, the use of Elastic Net regularization during the GPR learning phase allowed for automatic feature selection, enhancing the interpretability of the model for policy-makers by identifying the most relevant socioeconomic indicators.

Conclusion and Future Work

The integration of social data (mobile phones) and physical data (environment) provides a holistic view of poverty that surpasses single-source approaches. While the current framework is highly effective, future research will focus on extending factorized representations more robustly into the regression domain and exploring multi-output settings to predict various types of socioeconomic deprivations (e.g., health and education) simultaneously.

Summary updated

BibTeX
@inproceedings{pokhriyal2019multi,
  author = {Neeti Pokhriyal},
  title = {{Multi-View Learning from Disparate Sources for Poverty Mapping}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2019},
  doi = {10.1609/aaai.v33i01.33019892},
}
Figure