Abstract
In this paper, we develop a virtual laboratory for measuring human trust. Our laboratory, which is realized as a web application, enables researchers to show pre-recorded or live video feeds to groups of users in a synchronized fashion. Users are able to provide real-time feedback on these videos via affect buttons and a freeform chat interface. We evaluate our application via a quantitative user study ($N \approx 80$) involving videos of cyber-physical systems, such as autonomous vehicles, performing positively or negatively. Using data collected from user responses in the application, as well as customized survey instruments assessing different facets of trust, we find that human trust in cyber-physical systems can be affected merely by remotely observing the behavior of such systems, without ever encountering them in person.
Introduction
As autonomous technologies like self-driving taxis and industrial robots become more prevalent, the way humans form trust in these systems is shifting. Traditionally, trust research has focused on direct, hands-on interaction. However, in the modern digital landscape, first impressions are often formed through remote observation—via news segments, social media, or YouTube clips. This paper addresses the gap in understanding how trust evolves through passive, remote observation, exploring whether "seeing" truly leads to "believing" when it comes to complex, networked hardware.
The Virtual Laboratory
To investigate this phenomenon, we developed a custom Virtual Laboratory, a web-based platform designed for synchronized group observation. Unlike standard survey tools, this application allows researchers to stream high-definition video to multiple participants simultaneously, ensuring a shared experience.
The platform integrates two primary feedback mechanisms:
- Real-Time Affect Buttons: Users can log their emotional state (e.g., "Anxious," "Confident," "Satisfied") as events unfold on screen.
- Collaborative Chat: A freeform interface allows participants to respond to qualitative prompts and view the reactions of others, simulating a digital focus group environment.
Experimental Methodology
We conducted a two-session study spaced one week apart with approximately 80 participants. The study utilized curated video datasets representing three types of Cyber-Physical Systems (CPS): Autonomous Vehicles (AV), Power Plants (PP), and Robotic Assembly Lines (RAL). For the AV dataset, we presented two versions: one showing "Positive" (safe, predictable) behavior and another showing "Negative" (erratic, dangerous) behavior.
Trust was measured across three dimensions: dispositional (general tendency), situational (context-based), and learned (performance-based). We employed a "stealth" survey design, adapting established trust scales while removing the word "trust" to prevent priming and ensure responses were based on perceived behavior.
Key Insights and Results
Our findings indicate that remote observation is a powerful catalyst for trust adjustment. Key results include:
- Significant Shifts in Learned Trust: The group observing "Negative" AV behavior showed a statistically significant decrease in trust ($p < 0.001$), while the "Positive" group showed a significant increase ($p < 0.001$).
- The Contrast Effect: Surprisingly, participants who witnessed AV failures in the first session reported higher trust scores for Power Plants and Assembly Lines in the second session compared to those who saw positive AV behavior first. This suggests that witnessing failure can prime users to be more appreciative of subsequent systems that function correctly.
- Sentiment Correlation: Real-time sentiment analysis, performed using a fine-tuned DistilBERT model on chat logs, tracked closely with system performance. Sentiment in the "Negative" AV group dropped rapidly and remained low throughout the session.
- Domain Specificity: Trust did not transfer directly between different types of CPS. Participants maintained distinct mental models for self-driving cars versus industrial power plants, suggesting that trust is highly context-dependent.
Discussion and Future Work
While our study confirms that remote observation significantly impacts trust, several limitations remain. The participant pool was largely composed of university students (70%), who may be more tech-inclined than the general public. Additionally, the collaborative nature of the chat interface introduces potential social propagation effects, where peer reactions might influence individual trust levels.
Future research will explore the "Deepfake" problem—investigating how trust evolves when users suspect video evidence may be synthetic. Furthermore, we aim to compare the effect size of virtual observation against in-person interaction to determine the extent to which trust-building can be effectively digitized.
Summary updated