Detecting Distrust Towards the Skills of a Virtual Assistant Using Speech

Leonardo Pepino, Pablo Riera, Lara Gauder, Agustín Gravano, Luciana FerrerarXiv 2020 · arXiv

Abstract

This paper investigates the feasibility of detecting user distrust in virtual assistants (VAs) through acoustic speech analysis. By identifying "hyper-articulation" cues—similar to those used when speaking to children or non-native speakers—researchers developed a machine learning model that achieves 76% accuracy in distinguishing between interactions with reliable and unreliable assistants. The study highlights the importance of feature normalization and temporal context in capturing the subtle paralinguistic signals of skepticism.

Introduction

Trust is a fundamental component of human-computer interaction (HCI). Calibrating user trust is essential to prevent over-reliance on flawed systems or under-utilization of capable ones. Ideally, a Virtual Assistant should be able to sense a user's skepticism in real-time and adjust its behavior—perhaps by providing more detailed explanations or "proof" for its answers—to calibrate that trust.

While emotional states are known to influence vocal delivery, the specific "sound of distrust" in digital interactions remains under-explored. This research explores whether specific vocal features can serve as a reliable proxy for a user's internal level of trust, focusing on paralinguistic cues that emerge when a system's reliability is questioned.

Methodology

The researchers framed the detection of distrust as a binary classification task: distinguishing whether a user was speaking to a "Reliable" (High-trust) or "Unreliable" (Low-trust) assistant.

Feature Engineering and Normalization

Instead of using "black-box" deep learning, the team utilized 16 hand-crafted acoustic features inspired by "foreigner-directed" and "infant-directed" speech research. These features focus on hyper-articulation, including syllable rates, pitch (F0) excursions, energy ranges, and formant variations.

A critical technical contribution was the normalization approach. Vocal features are heavily influenced by individual identity and the specific question asked. By normalizing features by both user and question, the researchers significantly boosted model performance, as demonstrated by the reduction in normalized cross-entropy.

Impact of normalization and context on model performance.

Experimental Setup

The study utilized the Trust-UBA Database, involving 50 subjects in a laboratory setting. To manipulate trust, users were given prior ratings for the VA (High: 4.9/5 vs. Low: 1.4/5). In the "High" condition, the VA answered correctly, while in the "Low" condition, it intentionally made mistakes.

Key Insights: The Sound of Distrust

The research suggests that distrust sounds like "teaching." When users did not trust the VA, they subconsciously treated it like an "at-risk" listener (such as a child or non-native speaker). This resulted in slower speech (lower syllable rates) and higher pitch medians—classic signs of hyper-articulation intended to ensure comprehension.

Acoustic features (syllable rate and pitch) across high (H) and low (L) trust conditions for various users.

Furthermore, the model performed poorly when analyzing single questions but improved significantly when looking at a series of interactions. This indicates that distrust is a "slow-burn" signal that manifests over the course of an interaction rather than in a single moment.

Results and Discussion

The model achieved a peak 76% accuracy in detecting the reliability condition when using the best normalization strategy and series-level data. By narrowing the feature set to a "minimalist" trio—syllable rate (including pauses), pitch final slope, and pitch median—the researchers improved normalized cross-entropy by 38%, indicating that these three features capture the core of the distrust signal.

There was also a significant correlation (0.43) between the model’s confidence and the actual trust scores reported by users in surveys. This suggests that the model is indeed picking up on the subjective experience of distrust.

Limitations and Future Work

Despite these promising results, the model failed to perform better than random on data recorded remotely ("in-the-wild"). Background noise and hardware variations remain significant hurdles. Additionally, while VA reliability was used as a proxy for trust, future research should explore "unwarranted" distrust, where a user remains skeptical of a perfectly functional system. Larger and more culturally diverse datasets will be necessary to validate these preliminary findings across different languages and user groups.

Summary updated

Abstract

Research has shown that trust is an essential aspect of human-computer interaction directly determining the degree to which the person is willing to use the system. An automatic prediction of the level of trust that a user has on a certain system could be used to attempt to correct potential distrust by having the system take relevant actions like, for example, explaining its actions more thoroughly. In this work, we explore the feasibility of automatically detecting the level of trust that a user has on a virtual assistant (VA) based on their speech. We use a dataset collected for this purpose, containing human-computer speech interactions where subjects were asked to answer various factual questions with the help of a virtual assistant, which they were led to believe was either very reliable or unreliable. We find that the subject's speech can be used to detect which type of VA they were using, which could be considered a proxy for the user's trust toward the VA's abilities, with an accuracy up to 76\%, compared to a random baseline of 50\%. These results are obtained using features that have been previously found useful for detecting speech directed to infants and non-native speakers.
BibTeX
@misc{pepino2020detecting,
  author = {Leonardo Pepino and Pablo Riera and Lara Gauder and Agustín Gravano and Luciana Ferrer},
  title = {{Detecting Distrust Towards the Skills of a Virtual Assistant Using Speech}},
  year = {2020},
  eprint = {2007.15711},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2007.15711},
}
Figure