Abstract
Artificial Intelligence for Social Good (AI4SG) frequently encounters a "deployment gap" where theoretical models fail to translate into real-world impact. This paper addresses this gap by proposing a novel "bandit data-driven optimization" framework. By moving from static prediction to an iterative "predict-then-optimize" loop, the research demonstrates how AI can optimize complex social systems, such as food rescue operations. The proposed PROOF algorithm significantly improved volunteer engagement for the non-profit 412 Food Rescue, increasing the "hit rate" from 45% to 79%.
The Challenge of Real-World AI
The primary hurdle in AI4SG is the transition from prediction to prescription. While standard machine learning excels at predicting outcomes (e.g., "Will a volunteer claim this rescue?"), practitioners require actionable prescriptions (e.g., "Which specific volunteers should we notify to ensure the food is picked up without spamming everyone?").
The author identifies four critical barriers to this transition:
- Data Scarcity: Initial datasets are often insufficient for robust modeling.
- Endogeneity: Interventions, such as notifications, inherently alter the data distribution they are meant to model.
- Communication Gaps: Translating organizational goals into precise mathematical objectives is an iterative process.
- Unexpected Consequences: Real-world complexity often leads to outcomes that single-shot models cannot foresee.
Methodological Evolution: Closing the Loop
To overcome these hurdles, the paper outlines an evolution of optimization paradigms, culminating in a framework that treats deployment as a continuous learning process.
- Two-stage prediction-prescription: The traditional approach where a predictive model's output serves as the input for a separate optimization algorithm.
- One-shot data-driven optimization: An integrated approach where the optimization objective is incorporated into the machine learning model's loss function.
- Bandit data-driven optimization (Proposed): An iterative framework where the system implements a recommendation, collects resulting data, and updates the model in a continuous loop.
The PROOF Algorithm
The centerpiece of this research is the PROOF (PRedict-then-Optimize with Optimism in Face of uncertainty) algorithm. PROOF is a modular framework that bridges offline predictive analytics with online learning. By adopting an "optimistic" strategy, the algorithm explores potential interventions that might outperform current strategies. This approach ensures "no-regret" performance, meaning the system eventually performs as effectively as the best possible fixed strategy identified in hindsight.
Case Study: 412 Food Rescue
The framework was validated using real-world data from 412 Food Rescue, a non-profit coordinating food donations across 55 cities. Unlike traditional recommendation systems for movies or books, food rescue trips are unique, non-reusable events. To address this "cold start" problem, the author utilized contextual neural architectures to model volunteer behavior based on donation locations, volunteer history, and rescue trip outcomes.
Results and Impact
The application of the PROOF framework yielded significant improvements in operational efficiency:
- Efficiency Gains: The "hit rate"—the probability of a volunteer claiming a notified trip—rose from 45% to 79%.
- Real-World Adoption: 412 Food Rescue officially adopted the recommended intervention scheme in January 2020.
- Theoretical Robustness: Simulations confirmed that PROOF maintains superior performance and "no-regret" guarantees even when interventions shift the underlying data distribution.
Conclusion
This research highlights that AI4SG is not merely the application of existing tools to social problems, but a driver for new algorithmic research. By allowing real-world constraints like "notification fatigue" to shape research, we can develop robust systems like PROOF that bridge the gap between theory and practice. Future work aims to scale this framework to other domains, including public health and environmental sustainability.
Summary updated