Abstract
Social and behavioral interventions are a critical tool for governments and communities to tackle deep-rooted societal challenges such as homelessness, disease, and poverty. However, real-world interventions are almost always plagued by limited resources and limited data, which creates a computational challenge: how can we use algorithmic techniques to enhance the targeting and delivery of social and behavioral interventions? The goal of my thesis is to provide a unified study of such questions, collectively considered under the name "algorithmic social intervention". This proposal introduces algorithmic social intervention as a distinct area with characteristic technical challenges, presents my published research in the context of these challenges, and outlines open problems for future work. A common technical theme is decision making under uncertainty: how can we find actions which will impact a social system in desirable ways under limitations of knowledge and resources? The primary application area for my work thus far is public health, e.g. HIV or tuberculosis prevention. For instance, I have developed a series of algorithms which optimize social network interventions for HIV prevention. Two of these algorithms have been pilot-tested in collaboration with LA-area service providers for homeless youth, with preliminary results showing substantial improvement over status-quo approaches. My work also spans other topics in infectious disease prevention and underlying algorithmic questions in robust and risk-aware submodular optimization.
Introduction
Governments and NGOs regularly deploy interventions to tackle massive societal challenges like disease, poverty, and homelessness. However, these organizations face two crushing constraints: limited resources (not enough money or staff to reach everyone) and limited data (uncertainty about how people interact or who is most influential). Traditionally, deciding who to target for an outreach campaign or where to send medical supplies relies on manual intuition or simple heuristics. This research argues that these methods are inefficient and proposes a computational framework to find the specific actions that will have the greatest impact on a social system when the "map" of that system is not fully understood.
The Algorithmic Social Intervention Pipeline
The author proposes a unified pipeline for Algorithmic Social Intervention consisting of four stages, mapping specific technical contributions to the lifecycle of an intervention.
- Information Gathering: Efficiently acquiring the data needed to optimize an intervention (e.g., surveying a social network to find connections).
- Optimization under Uncertainty: Using robust and risk-aware submodular optimization. Submodularity is a mathematical property representing "diminishing returns"—the idea that the 100th person you recruit for a program adds less value than the 1st. The author developed algorithms that work even when the objective function is not fully known.
- Adaptive Sequential Decision Making: Adjusting the strategy in real-time as the intervention progresses and new data comes in.
- Field Evaluation: Moving beyond simulations to test algorithms in the real world alongside non-profit partners.
Key Insights and Methodology
The technical "secret sauce" lies in handling robustness. Instead of assuming the data is perfect, the algorithms are designed to perform well even in "worst-case" scenarios where network connections might be missing or behaviors might change. Key insights include:
- Embracing Irrationality: Unlike classic game theory, which assumes people are perfectly rational, this framework acknowledges that human behavior is messy and requires models from social and behavioral sciences.
- Robustness over Accuracy: In the field, a "highly accurate" model that is fragile is less useful than a "robust" model that performs decently across a wide range of uncertainties.
- Closing the Loop: Theoretical guarantees and simulations are insufficient; the "gold standard" is validating the algorithm through field experiments where it competes against existing human-led policies.
Results and Field Impact
The research utilizes real-world data gathered through collaborations, such as social network data from LA-area drop-in centers and epidemiological data for Tuberculosis (TB) in India. Key results include:
- HIV Prevention: In pilot tests with homeless youth in Los Angeles, the AI-driven algorithms resulted in over twice as many youth receiving HIV prevention messages compared to the status-quo heuristic used by service providers.
- Tuberculosis Control: In simulations based on Indian public health data, the proposed algorithm was estimated to avert over 8,000 cases of TB per year compared to current policies.
- Algorithmic Efficiency: The research successfully formalized "information gathering" as a way to minimize the time-intensive process of surveying social networks while still maintaining high intervention impact.
Conclusion and Future Work
While the framework has shown significant success, several hurdles remain. Social interventions are often deployed in "data deserts" where basic information is missing, and human agents are not perfectly predictable. Furthermore, conducting field experiments is arduous and time-consuming. Future work remains in refining "adaptive" strategies—how to best change course mid-intervention when the initial data turns out to be wrong—and scaling these solutions to broader societal challenges.
Summary updated