Exploring Causality for HRI: A Case Study on Robotic Mental Well-being Coaching

Micol Spitale, Srikar Babu, Serhan Cakmak, Jiaee Cheong, Hatice Gunes

TL;DR

This paper explores how causal reasoning—moving beyond simple pattern recognition to understanding "cause and effect"—can improve how robots adapt during mental well-being coaching. By applying macro-level structural modeling and micro-level causal inference to a 4-week longitudinal study, the researchers identified specific non-verbal cues (like speech loudness and specific facial muscle movements) that directly influence a user's perception of the robot and the overall success of the coaching.

What Problem Does This Paper Address?

Adaptive robots in Human-Robot Interaction (HRI) typically rely on machine learning models like Reinforcement Learning (RL). However, these models face a "data hunger" problem: in sensitive fields like mental health, real-world data is scarce and expensive to collect.

Furthermore, standard ML models are often "black boxes" that find correlations but don't understand the underlying relationships. If a user stops talking, is it because they are reflecting (good) or because the robot was intrusive (bad)? Without understanding causality, robots struggle to generalize their behavior to new users or explain why they made a specific decision. This paper investigates how causal discovery tools can bridge this gap by mapping the interplay between human behavior and robotic response.

Methods

The researchers employed a two-tiered causal analysis approach:

  1. Macro-level Analysis (Structural Equation Modeling - SEM): This looked at the "big picture" across entire coaching sessions. SEM was used to model how latent constructs (like a user's overall perception of the robot) are influenced by aggregated features like average speech pitch or facial expressions.
  2. Micro-level Analysis (Fast Causal Inference - FCI): This focused on the "nitty-gritty" of turn-by-turn interactions. The FCI algorithm was used to discover causal structures in high-dimensional data, helping to identify which specific non-verbal cues at moment $t$ caused a specific reward or state change at moment $t+1$.
  3. Perception & Affect Tools: They used OpenFace for facial action units (FAUs) and openSMILE for audio features (loudness, pitch, etc.).

Datasets

The study utilized the VITA dataset, which originated from a previous longitudinal study:

  • Participants: 17 coachees in a workplace setting.
  • Duration: 4 weeks of positive psychology exercises delivered by a QTrobot.
  • Metrics:
    • PANAS: Positive and Negative Affect Schedule (emotional state).
  • ROSAS: Robotic Social Attributes Scale (perception of the robot's warmth/competence).
  • WAI-SR: Working Alliance Inventory (the "bond" between human and robot).
  • Features: 17 facial features and 7 audio features were extracted from the interaction recordings.

Key Insights and Ideas

  • The "Clarity" Connection: At the macro level, speech loudness and spectral flux (variability in speech speed) were positively correlated with better perceptions of the robot (ROSAS). Essentially, when users spoke more clearly and dynamically, they tended to view the robot more favorably.
  • Tension as a Barrier: Specific facial cues, particularly AU7 (Lid Tightening), showed a strong negative causal relationship with both the user's perception of the robot and the therapeutic alliance (WAI). This suggests that facial expressions associated with tension or squinting are primary indicators of a failing interaction.
  • Structural Separation in Affect: The micro-analysis (FCI) revealed a clear "structural separation" in the causal graph. Negative emotional cues (like AU15 - lip corner depressor) clustered together, while positive cues (AU6/AU12 - cheek raiser/lip corner puller) formed a separate cluster, providing a roadmap for how a robot might "steer" an interaction toward the positive cluster.
  • Speech Duration as a Reward Proxy: The analysis confirmed that speech duration is a valid component for reinforcement learning rewards, as it showed a potential causal link with the robot's internal reward calculation.

Key Results

  • Audio Impact: Loudness had the strongest positive path coefficient for robot perception (0.681), followed by Spectral Flux (0.439).
  • Facial Impact: AU7 (Lid Tightening) had a massive negative impact on perception (-1.014) and a significant negative impact on the therapeutic bond (-0.501).
  • Model Fit: The SEM models showed high Comparative Fit Index (CFI) values and low Root Mean Square Error of Approximation (RMSEA), validating that the identified causal relationships were statistically sound.

Limitations and Future Work

  • The "Counterfactual" Problem: A major limitation in causal inference is that we only observe what did happen (the factual), not what would have happened if the robot had acted differently (the counterfactual).
  • Methodological Scope: The study primarily relied on Maximum Likelihood Estimation (MLE) to quantify effects. The authors suggest that future work should explore other measures to quantify causal effects.
  • Generalization: While the case study on mental well-being is robust, the authors aim to extend this methodology to other HRI domains, such as collaborative manufacturing or elderly care, to see if these causal patterns hold true across different contexts.

Summary updated

Abstract

One of the primary goals of Human-Robot Interaction (HRI) research is to develop robots that can interpret human behavior and adapt their responses accordingly. Adaptive learning models, such as continual and reinforcement learning, play a crucial role in improving robots' ability to interact effectively in real-world settings. However, these models face significant challenges due to the limited availability of real-world data, particularly in sensitive domains like healthcare and well-being. This data scarcity can hinder a robot's ability to adapt to new situations. To address these challenges, causality provides a structured framework for understanding and modeling the underlying relationships between actions, events, and outcomes. By moving beyond mere pattern recognition, causality enables robots to make more explainable and generalizable decisions. This paper presents an exploratory causality-based analysis through a case study of an adaptive robotic coach delivering positive psychology exercises over four weeks in a workplace setting. The robotic coach autonomously adapts to multimodal human behaviors, such as facial valence and speech duration. By conducting both macro- and micro-level causal analyses, this study aims to gain deeper insights into how adaptability can enhance well-being during interactions. Ultimately, this research seeks to advance our understanding of how causality can help overcome challenges in HRI, particularly in real-world applications.
BibTeX
@inproceedings{spitale2025exploring,
  author = {Micol Spitale and Srikar Babu and Serhan Cakmak and Jiaee Cheong and Hatice Gunes},
  title = {{Exploring Causality for HRI: A Case Study on Robotic Mental Well-being Coaching}},
  booktitle = {RO-MAN},
  year = {2025},
  doi = {10.1109/ro-man63969.2025.11217880},
}

Loading related content...

Figure