Abstract
This paper investigates the specific experience of following a suggestion by an intelligent machine that has a wrong outcome and the emotions people feel. By adopting a typical task employed in studies on decision-making, we presented participants with two scenarios in which they follow a suggestion and have a wrong outcome by either an expert human being or an intelligent machine. We found a significant decrease in the perceived responsibility on the wrong choice when the machine offers the suggestion. At present, few studies have investigated the negative emotions that could arise from a bad outcome after following the suggestion given by an intelligent system, and how to cope with the potential distrust that could affect the long-term use of the system and the cooperation. This preliminary research has implications in the study of cooperation and decision making with intelligent machines. Further research may address how to offer the suggestion in order to better cope with user’s self-blame.
Introduction
As AI-driven recommendation engines and decision-support systems become ubiquitous—guiding everything from consumer purchases to healthcare management—Human-Computer Interaction (HCI) research must look beyond technical accuracy to the emotional aftermath of system errors. While extensive literature exists regarding user trust in AI, there is a notable gap in understanding how users feel when that trust is betrayed by a faulty suggestion.
This study specifically investigates the phenomenon of "self-blame." When a user follows a machine’s bad advice, do they internalize the failure to the same degree as they would if a human expert had provided the same suggestion? Understanding this emotional dynamic is critical for designing systems that foster healthy cooperation without inducing reckless over-reliance or a total breakdown of trust following inevitable errors.
Methods
The researchers employed a scenario-based (vignette) approach, a standard practice in decision-making psychology to maintain tight control over experimental variables. Participants were recruited via Amazon Mechanical Turk and randomly assigned to one of two conditions:
- The Human Condition: Participants imagined purchasing a smartphone based on the recommendation of a human shop clerk, which ultimately failed to meet their needs.
- The Computer Condition: Participants imagined the same scenario, but the suggestion originated from an "algorithm-based website."
Following the scenarios, participants rated their feelings of regret, disappointment, guilt, and perceived responsibility on a 10-point Likert scale. The study also included a longitudinal element, checking back in after a hypothetical year to see how these emotions evolved.
Initially, 85 participants were recruited. To ensure data integrity, the authors applied rigorous quality filters, removing participants who provided inconsistent answers or completed the task too quickly (using Tukey's fences for outlier detection). This resulted in a final sample of 38 high-quality participants for the primary analysis.
Key Results
The study revealed a significant "responsibility buffer" effect when interacting with intelligent systems. While users still felt negative emotions regarding the poor outcome, the source of the advice fundamentally changed their perception of personal fault.
- Perceived Responsibility: In the initial scenario, participants reported significantly less self-blame in the computer condition (Mean = 6.6) compared to the human condition (Mean = 8.4). This difference remained statistically significant in the follow-up scenario (p = 0.043).
- Variance in AI Perception: The "Computer" group exhibited much higher variance in their responses (p = 0.006). This suggests that users lack a unified mental model for AI; some may view it as an infallible authority, while others see it as a neutral, non-agentic tool.
- Source-Agnostic Emotions: Interestingly, there were no statistically significant differences found for regret, disappointment, or guilt between the two groups. Users feel equally bad about the outcome itself, regardless of whether a human or a machine led them there; they simply feel less personally responsible for the choice when a machine is involved.
Discussion and Conclusion
The findings suggest that because machines currently lack perceived agency or intent, they serve as a shield against self-blame. While this might make the experience of a bad outcome less emotionally taxing for the user, it carries a hidden risk: if users feel less responsible for AI-assisted decisions, they may engage in riskier behavior or exhibit over-reliance on the system.
This preliminary work highlights the need for designers to find a balance. Systems should be designed to keep users critically engaged in the decision-making process to ensure they maintain a sense of agency and responsibility. Future research should expand beyond vignettes to real-time interactions with AI systems and explore how personality traits, such as Locus of Control, influence these emotional reactions.
Summary updated