Steps Towards Satisficing Distributed Dynamic Team Trust

Edmund R. Hunt, Chris Baber, Mehdi Sobhani, Sanja Milivojevic, Sagir Muhammad Yusuf, Mirco Musolesi, Patrick Waterson, Sally MaynardarXiv 2023 · DOI · OpenReview

TL;DR

This paper proposes a novel framework for defining and measuring "satisficing trust" in dynamic human-robot teams. They argue that trust can be understood through a hierarchy of goals, values, and legal principles, and introduce interpretable metrics for both humans and robots to quantify trust, paving the way for more robust human-robot collaboration.

What Problem Does This Paper Address?

The core challenge this paper tackles is the lack of a clear, quantifiable, and interpretable definition of "trust" in dynamic human-robot teams. In critical domains like defense and security, it's crucial for team members (both human and robot) to trust each other to work towards shared objectives and adhere to common principles. Existing approaches often rely on human-centric perceptions (like self-report questionnaires) that are not directly applicable to robots. Furthermore, the dynamic nature of team missions means trust levels need to adapt to changing situations, leading to a concept of "satisficing trust" – achieving a minimally acceptable level of trust to progress. The paper aims to bridge this gap by proposing a unified framework for defining and measuring trust that is meaningful for both human and robot agents.

Methods

The authors propose a hierarchical model of trust, breaking it down into three key components: Capability, Predictability, and Integrity. They argue that while alignment on goals and legal principles is achievable between humans and robots, alignment on subjective values is more complex and likely remains within the human domain.

To operationalize these components, they introduce a set of metrics that can be measured or inferred from agent actions and interactions. These metrics are then integrated into a Bayesian Belief Network (BBN) model. This BBN allows agents to dynamically update their estimates of a teammate's capability, predictability, and integrity based on observed actions and contextual information. The "why" behind an action (related to values) is particularly important for assessing integrity, especially when considering potential conflicts between individual and team goals. The concept of a "ladder of trust" is used as a metaphor to describe how these trust estimates can fluctuate during a mission.

Datasets

Not discussed in the paper. The paper focuses on a theoretical framework and proposes an experimental setup for future validation.

Key Insights and Ideas

  • Hierarchy of Trust Components: Trust isn't monolithic; it's built upon measurable components of capability, predictability, and integrity.
  • Goal/Principle Alignment vs. Value Alignment: While robots can be programmed to align with team goals and legal principles, aligning with nuanced human values is a significant challenge.
  • "Satisficing Trust": Trust isn't about absolute certainty but about reaching a sufficient level for the current situation to enable mission progress.
  • Dynamic and Contextual Trust: Trust is not static; it evolves based on ongoing interactions and the specific context of the mission.
  • Interpretable Metrics for Robots and Humans: The proposed metrics are designed to be understandable and usable by both human and robot agents, facilitating mutual understanding.
  • Integrity as Action-Based: Integrity is defined by the observed choice of action in a given context, rather than an inherent trait, allowing for dynamic assessment and reputation building.

Key Results

The paper does not present empirical results as it outlines a theoretical framework and proposes an experimental design. The key "results" are conceptual:

  • A formal definition of trust components (Capability, Predictability, Integrity) applicable to human-robot teams.
  • A proposed hierarchy of goals, values, and legal principles as the foundation for trust.
  • A Bayesian Belief Network (BBN) model for dynamically updating trust estimates.
  • A conceptual experiment designed to demonstrate "satisficing trust" and the "ladder of trust" metaphor.

Limitations and Future Work

The authors acknowledge that their proposed framework is theoretical and requires empirical validation.

Limitations: * The paper does not empirically validate the proposed metrics or the BBN model. * The complexity of inferring "values" and their influence on integrity, especially for robots, remains a challenge. * The assumption that humans and robots can interpret each other's BBNs needs further investigation.

Future Work: * Conducting experiments to test the proposed framework and metrics in simulated or real-world scenarios. * Developing more sophisticated methods for robots to infer human values and for humans to understand robot decision-making processes. * Investigating the impact of factors like overtrust and undertrust on team performance. * Exploring how to integrate long-term trust dynamics and relationship equity into the BBN model.

Summary updated

Abstract

Defining and measuring trust in dynamic, multiagent teams is important in a range of contexts, particularly in defense and security domains. Team members should be trusted to work towards agreed goals and in accordance with shared values. In this paper, our concern is with the definition of goals and values such that it is possible to define 'trust' in a way that is interpretable, and hence usable, by both humans and robots. We argue that the outcome of team activity can be considered in terms of 'goal', 'individual/team values', and 'legal principles'. We question whether alignment is possible at the level of 'individual/team values', or only at the 'goal' and 'legal principles' levels. We argue for a set of metrics to define trust in human-robot teams that are interpretable by human or robot team members, and consider an experiment that could demonstrate the notion of 'satisficing trust' over the course of a simulated mission.
BibTeX
@misc{hunt2023steps,
  author = {Edmund R. Hunt and Chris Baber and Mehdi Sobhani and Sanja Milivojevic and Sagir Muhammad Yusuf and Mirco Musolesi and Patrick Waterson and Sally Maynard},
  title = {{Steps Towards Satisficing Distributed Dynamic Team Trust}},
  year = {2023},
  eprint = {2309.05378},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2309.05378},
}
Figure