Blending Participatory Design and Artificial Awareness for Trustworthy Autonomous Vehicles

Ana Tanevska, Ananthapathmanabhan Ratheesh Kumar, Arabinda Ghosh, Ernesto Casablanca, Ginevra Castellano, Sadegh Soudjani

Introduction

Current research in Multi-Agent Systems (MAS) has made significant strides in coordinating robotic agents. However, these systems often struggle to account for the "human in the loop." In complex urban environments, autonomous vehicles (AVs) must interact with pedestrians, cyclists, and human drivers whose behaviors are non-deterministic and heavily influenced by trust and perceived risk.

Proposed architecture for an autonomous agent incorporating situational awareness and communication.

The core challenge lies in the lack of robust, data-driven models of human behavior that can be integrated into an AV's "Artificial Awareness" architecture. This paper addresses this gap by investigating how transparency (information sharing) and agency (control retention) influence the human-AV relationship, ultimately providing a mathematical framework to help AVs predict human actions and foster trust.

Methodology

The researchers employed a multi-stage, participatory approach to bridge technical MAS architectures with human-centered design.

Participatory Design and User Study

Initially, a participatory design workshop was held to develop two distinct AV interfaces: a "Low Information" minimalist GPS-style display and a "High Information" interface providing detailed visualizations of sensor data and planned actions. Following this, a large-scale user study was conducted with 206 participants. Participants were presented with simulated driving scenarios—a highway cut-in and an unexpected slowing in a suburban area—and asked to choose their preferred Level of Autonomy (LoA) at various intervals.

The six levels of vehicle autonomy, ranging from full human control to full machine automation.

Markov Chain Modeling

The behavioral data from 1,236 specific action choices was used to construct Markov Chain models. In these models, states represent the driver's preferred LoA: Takeover (manual control), Alert (assisted driving), and Normal (highly automated). By calculating transition probabilities between these states, the authors could mathematically describe how driver behavior shifts as a scenario unfolds.

Results and Discussion

The study revealed several critical insights into the dynamics of human-AV interaction:

  • The Fragility of Trust: The "Takeover" state was found to be an "absorbing state." Once a participant decided to take manual control during a stressful event, there was a 75–100% probability they would not return control to the AI for the remainder of that interaction. This highlights that trust, once lost, is extremely difficult to regain in real-time.
  • Environmental Context: The driving environment had a more significant impact on behavior than the interface design. Transition patterns varied significantly between highway and suburban settings ($\chi^2 = 39.76, p < 0.001$). In suburbs, the "Normal" automated state was highly stable (68–80% probability), whereas on the highway, stability dropped to 29–35%.
  • Demographic Variations: The study found statistically significant differences in transition patterns based on participant sex ($\chi^2 \approx 6.1, p < 0.05$), suggesting that personalized driver models may be necessary for safety-critical systems.
  • Transparency Effects: While environment was the dominant factor, high transparency (the "High Information" interface) did reduce the probability of a "Takeover" in specific highway transitions (33.3% vs. 51.5% for Low Info).

Conclusion and Future Work

By creating data-driven Markov models of human behavior, this research provides a foundation for more "aware" autonomous systems. Future work will focus on programming these Markov models into tools like PRISM to verify safety properties and incorporating the human driver model into the SymAware MAS architecture to enable real-time, adaptive risk assessment. This approach ensures that as AVs become more sophisticated, they remain grounded in human needs and predictable to the people they serve.

Summary updated

BibTeX
@inproceedings{tanevska2025blending,
  author = {Ana Tanevska and Ananthapathmanabhan Ratheesh Kumar and Arabinda Ghosh and Ernesto Casablanca and Ginevra Castellano and Sadegh Soudjani},
  title = {{Blending Participatory Design and Artificial Awareness for Trustworthy Autonomous Vehicles}},
  booktitle = {RO-MAN},
  year = {2025},
  doi = {10.1109/ro-man63969.2025.11217774},
}
Figure