Abstract
In human-agent interactions where goals conflict, providing effective advice is a significant challenge. This paper explores "choice selection processes" with ranked options, exemplified by an "AC Game" where an agent aims for energy efficiency while a human seeks comfort. We propose a Social Utility Approach (SAP) that models human behavior using hyperbolic discounting and logit quantal response. Experimental results from a study with 272 participants demonstrate that the SAP method significantly outperforms traditional Markov Decision Process (MDP) models and baseline strategies in balancing energy savings with user acceptance.
Introduction
Traditional advice-giving agents often operate in environments with unordered choices. However, many real-world scenarios involve ranked options (e.g., setting a thermostat or volume level). This study addresses the "AC Game," characterized by ordered actions, partial acceptance (users "meeting the agent halfway"), partial observability of the environment, and conflicting costs between the agent (energy) and the human (comfort). The core challenge is providing advice over multiple rounds that minimizes energy consumption without eroding user trust.
Methodology
The study compares several advising strategies:
- Social Utility Approach (SAP): This model optimizes a weighted combination of agent and human costs. It incorporates a sophisticated human model featuring hyperbolic discounting—accounting for the human tendency to prioritize immediate comfort—and logit quantal response to model noisy, "mostly rational" decision-making.
- Markov Decision Process (MDP): A standard sequential decision-making framework. To maintain computational feasibility, the human model was simplified using exponential smoothing, assuming users eventually learn their exact preferences.
- Baselines: These include Silent (no advice), Receiver-focused (maximizing human comfort), and Sender-focused (maximizing energy savings).
The researchers evaluated these methods through an empirical study on Amazon Mechanical Turk (AMT) involving 272 participants who played the AC Game, receiving advice on power levels while attempting to maximize their personal score.
Key Results
The empirical results highlight the superiority of the SAP method across multiple metrics.
As shown in Table 1, the SAP method achieved the lowest energy consumption (3.952), outperforming both the MDP (4.361) and the Sender-focused baseline (4.437). Notably, SAP maintained a high user acceptance rate of 34.5%, nearly matching the 36.7% rate of the Receiver-focused strategy, which prioritizes user comfort exclusively. Statistical analysis confirmed that SAP significantly outperformed all other methods (p < 0.001) in overall utility for the agent.
Discussion
The findings reveal an "optimality trap": while MDPs are theoretically optimal, their performance suffered because the human behavior model had to be oversimplified for mathematical solvability. In contrast, SAP’s robust modeling of complex human behavior proved more effective in practice.
Furthermore, the ranked nature of the options allowed the agent to use advice as a "nudge." Even when users did not follow advice perfectly, the agent could influence them toward more efficient settings. By balancing goals, the SAP agent preserves "trust" as a resource, maintaining enough influence to guide users toward energy efficiency over the long term.
Conclusion and Future Work
While the SAP approach shows great promise, its effectiveness depends on correctly weighting social utility components. Future research should explore how these weights can be determined in non-experimental settings and how models can account for even noisier human behavior in more complex, real-world environments.
Summary updated