Abstract
Many governmental bodies are adopting AI policies for decision-making. In particular, Reinforcement Learning (RL) has been used to design policies that citizens would be expected to follow. Much RL work assumes that citizens follow these policies unconditionally. However, without institutional trust, citizens may refuse to comply with government-mandated policies. This work develops a trust-aware RL algorithm for resource allocation in communities, specifically in humanitarian engineering contexts. Using a Deep Deterministic Policy Gradient (DDPG) approach, we model the feedback loop between an organization’s actions and community trust. We investigate how incorporating trust affects outcomes and how effectively an organization can learn policies when trust values are private. Our findings suggest that incorporating trust leads to more successful policies, particularly when organizational goals are uncertain. We also find that conservative trust estimates can increase fairness and average community trust, though at a cost to organizational success. Finally, we explore external interventions like quota systems to mitigate unfair outcomes.
Introduction
As governments increasingly turn to AI for high-stakes decisions like resource allocation, taxation, and pandemic response, they face a significant "compliance gap." If citizens do not trust the institution or the algorithm, they may refuse to follow the policy—for instance, by ignoring social distancing mandates or refusing a humanitarian resource.
Current RL research typically optimizes for specific organizational goals, such as cost minimization, while assuming perfect adoption. This paper argues that this is a dangerous oversight. The researchers set out to model the feedback loop between an organization’s actions and a community’s trust, asking whether an RL agent can learn to allocate resources effectively without eroding the trust required for the policy to function.
Methodology
The researchers developed a framework for Trust-Aware Reinforcement Learning applied to a humanitarian engineering scenario, such as distributing solar panels to a community.
The RL Agent and Trust Dynamics
The study utilizes a Deep Deterministic Policy Gradient (DDPG) approach, suitable for the continuous action space of determining "service quality" for individuals. Trust is modeled as a dynamic scalar that updates after every interaction. A citizen’s trust increases if they receive high utility and perceive their neighborhood as being treated fairly, measured via a local Gini coefficient.
Model Variants
The study compares three distinct model variants to understand the impact of trust information on policy learning. These variants differ in the scope of information available to the RL agent during the learning process.
- Trust-unaware: A baseline model that ignores trust entirely, assuming perfect compliance.
- Trust-aware: The agent has perfect knowledge of every individual's current trust levels and the underlying trust update rules.
- Learned-trust: The agent does not know trust values but uses Bayesian updates (Beta distributions) to estimate them based on whether citizens accept or reject services.
Experimental Setup
The study relies on simulated social networks generated to create realistic 15-node "seed" communities where edges represent information flow. Initial trust values were drawn from various Beta distributions to simulate different community "moods," ranging from highly skeptical to highly trusting. The researchers also tested a "quota system" intervention where the organization is penalized if it fails to serve at least half of the community.
Key Insights and Results
The experiments revealed several critical tensions between organizational efficiency and community well-being:
- The Efficiency-Trust Tension: When the organization prioritizes saving resources (a high "desire to save"), it inevitably erodes community trust. The RL agent becomes "successful" at saving money but fails to maintain the community's cooperation.
- The Information Paradox: Surprisingly, the Learned-trust model often produced fairer and more trust-improving policies than the model with perfect information. Because the Bayesian estimates were "conservative," the agent was less likely to take extreme, unfair actions that might alienate community members.
- Knowledge and Marginalization: In the Trust-aware model, the agent sometimes used its perfect knowledge to stop serving low-trust individuals entirely to maximize its own utility, effectively "giving up" on them and worsening inequality.
- Network Influence: High-degree nodes (individuals with many connections) are more susceptible to losing trust. Because they observe more of their neighbors' outcomes, they have a higher probability of witnessing unfair distributions, which sours their own perception of the institution.
- Quota Impact: Implementing a service quota successfully improved fairness and trust in scenarios with moderate resource-saving goals, though it reduced the organization's overall "success" metric by forcing higher expenditure.
Conclusion
This work underscores the critical importance of institutional trust in algorithm design and implementation. The primary limitation is the scale of the simulation; a 15-node environment is a "toy" model compared to real-world complexities. Furthermore, trust is a multi-faceted social construct influenced by historical marginalization and political factors beyond a simple scalar.
Future work should involve human-in-the-loop studies to refine trust update equations and testing the model on larger, real-world datasets. Exploring "collective action," where community members coordinate their responses to algorithmic decisions, remains a vital area for further research. Ultimately, this study identifies a fundamental tension between organizational success and community well-being that must be addressed in future AI policy design.
Summary updated