When we evaluate an AI system, we usually reach for aggregate metrics like accuracy, F1 score, or BLEU. However, research into human-AI interaction suggests that these numbers fail to capture how users actually experience technology. To a human, a system is not just a collection of data points; it is an agent with a perceived "capability" that users constantly re-evaluate.
This evaluation process is not linear or symmetric. As researchers like Harold Soh and Keyon Vafa have shown, human trust is a dynamic, task-dependent mapping that is far easier to destroy than it is to build.
The Human Generalization Function
Why does a single nonsensical recommendation lead a user to uninstall an app, even if the previous ten suggestions were perfect? In Frappe: Understanding the Usage and Perception of Mobile App Recommendations In-The-Wild, Linas Baltrunas and colleagues found that "high-severity" errors—such as suggesting a period-tracking app to a male user—disproportionately damage trust.
This happens because humans use what Keyon Vafa calls the Human Generalization Function. We build a mental model of what an AI should be able to do. When a system makes a "stupid" error on a simple task, it doesn't just count as one failure; it signals to the user that the system lacks a fundamental understanding of the world. This creates an update asymmetry: a single intuitive failure diminishes confidence more drastically than multiple successes can bolster it.
The Difficulty Gradient in Trust Transfer
Trust is not a single "global" value. Instead, it "leaks" between tasks based on perceived difficulty. In Multi-task trust transfer for human–robot interaction, Yaqi Xie and Harold Soh demonstrate that this transfer is inherently asymmetric:
- Top-Down Boost: If a robot successfully performs a high-difficulty task (like navigating a complex room), a human’s trust in its ability to perform easier tasks (like picking up a cup) increases significantly.
- Bottom-Up Skepticism: Conversely, demonstrating competence in simple tasks does not necessarily lead humans to trust the system with complex operations.
- The Penalty of Simple Errors: As Abed K. Musaffar notes in Learning to Lie: Reinforcement Learning Attacks Damage Human-AI Teams and Teams of LLMs, humans are much harsher on AI than on other humans. An error on a simple task by an AI causes a collapse in credibility that a human peer would not face for the same mistake.
The Recovery Lag
Once trust is broken, can it be fixed? The evidence suggests a significant "recovery lag." In A Diachronic Perspective on User Trust in AI under Uncertainty, Shehzaad Dhuliawala and Mrinmaya Sachan found that even after a system returns to high accuracy and perfect calibration, user skepticism persists.
This "long memory" for failure means that a brief period of high-confidence errors—where the system is wrong but claims to be certain—causes lasting damage to the human-AI loop. Even if the system becomes more reliable later, the user may continue to ignore its suggestions, leading to a permanent breakdown in collaboration.
Why Intuition Matters for Design
Understanding these asymmetries changes how we should build and deploy AI:
- Fluency vs. Adequacy: Marianna J. Martindale and Marine Carpuat found that users are often oblivious to "misleading" (fluent but wrong) translations but lose trust immediately over "disfluent" (clunky) text. This suggests that surface-level polish acts as a proxy for competence in the human mind.
- Adversarial Vulnerability: Because trust is so predictable and asymmetric, it can be weaponized. Abed K. Musaffar showed that adversarial agents can "learn to lie"—strategically failing or succeeding to manipulate human trust and maximize damage to a team's performance.
Ultimately, the goal of AI development shouldn't just be higher accuracy, but better trust calibration. If a system can signal its own boundaries and avoid "stupid" errors on simple tasks, it can prevent the catastrophic collapses in trust that currently hinder human-AI partnership.