Exploring Causality for HRI: A Case Study on Robotic Mental Well-being Coaching

Micol Spitale, Srikar Babu, Serhan Cakmak, Jiaee Cheong, Hatice Gunes

Abstract

One of the primary goals of Human-Robot Interaction (HRI) research is to develop robots that can interpret human behavior and adapt their responses accordingly. Adaptive learning models, such as continual and reinforcement learning, play a crucial role in improving robots' ability to interact effectively in real-world settings. However, these models face significant challenges due to the limited availability of real-world data, particularly in sensitive domains like healthcare and well-being. This data scarcity can hinder a robot's ability to adapt to new situations. To address these challenges, causality provides a structured framework for understanding and modeling the underlying relationships between actions, events, and outcomes. By moving beyond mere pattern recognition, causality enables robots to make more explainable and generalizable decisions. This paper presents an exploratory causality-based analysis through a case study of an adaptive robotic coach delivering positive psychology exercises over four weeks in a workplace setting. The robotic coach autonomously adapts to multimodal human behaviors, such as facial valence and speech duration. By conducting both macro- and micro-level causal analyses, this study aims to gain deeper insights into how adaptability can enhance well-being during interactions. Ultimately, this research seeks to advance our understanding of how causality can help overcome challenges in HRI, particularly in real-world applications.

Introduction

Adaptive robots in Human-Robot Interaction (HRI) typically rely on machine learning models like Reinforcement Learning (RL). However, these models face a "data hunger" problem: in sensitive fields like mental health, real-world data is scarce and expensive to collect. Furthermore, standard ML models are often "black boxes" that find correlations but don't understand underlying relationships. This paper investigates how causal reasoning—moving beyond simple pattern recognition to understanding "cause and effect"—can improve how robots adapt during mental well-being coaching.

Methodology

The researchers employed a two-tiered causal analysis approach to map the interplay between human behavior and robotic response:

  1. Macro-level Analysis (Structural Equation Modeling - SEM): This looks at the "big picture" across entire coaching sessions. SEM was used to model how latent constructs (like a user's overall perception of the robot) are influenced by aggregated features like average speech pitch or facial expressions.
  2. Micro-level Analysis (Fast Causal Inference - FCI): This focused on the "nitty-gritty" of turn-by-turn interactions. The FCI algorithm was used to discover causal structures in high-dimensional data, helping to identify which specific non-verbal cues at moment $t$ caused a specific reward or state change at moment $t+1$.
Overview of the macro-level analysis across weeks and micro-level analysis across interaction turns.

Dataset and Metrics

The study utilized the VITA dataset, originating from a longitudinal study involving 17 coachees in a workplace setting interacting with a QTrobot over four weeks. Perception and affect were tracked using OpenFace for facial action units (FAUs) and openSMILE for audio features.

Summary of the psychological measures used to evaluate emotional state, robot perception, and therapeutic alliance.

Key metrics included:

  • PANAS: Positive and Negative Affect Schedule (emotional state).
  • ROSAS: Robotic Social Attributes Scale (perception of warmth/competence).
  • WAI-SR: Working Alliance Inventory (the therapeutic bond).

Key Insights and Results

  • The "Clarity" Connection: At the macro level, speech loudness and spectral flux (variability in speech speed) were positively correlated with better perceptions of the robot (ROSAS). Loudness had the strongest positive path coefficient for robot perception (0.681).
  • Tension as a Barrier: Specific facial cues, particularly AU7 (Lid Tightening), showed a strong negative causal relationship with both the user's perception of the robot (-1.014) and the therapeutic alliance (-0.501). This suggests that facial expressions associated with tension are primary indicators of a failing interaction.
  • Structural Separation in Affect: Micro-analysis (FCI) revealed a clear "structural separation" in the causal graph. Negative emotional cues (like AU15 - lip corner depressor) clustered together, while positive cues (AU6/AU12 - cheek raiser/lip corner puller) formed a separate cluster.
  • Speech Duration: The analysis confirmed that speech duration is a valid component for reinforcement learning rewards, as it showed a potential causal link with the robot's internal reward calculation.

Conclusion

While this study demonstrates the utility of causal discovery in HRI, it is limited by the "counterfactual problem"—the inability to observe what would have happened if the robot had acted differently. Future work aims to extend this methodology to other domains, such as collaborative manufacturing or elderly care, to verify if these causal patterns hold across different contexts.

Summary updated

Abstract

One of the primary goals of Human-Robot Interaction (HRI) research is to develop robots that can interpret human behavior and adapt their responses accordingly. Adaptive learning models, such as continual and reinforcement learning, play a crucial role in improving robots' ability to interact effectively in real-world settings. However, these models face significant challenges due to the limited availability of real-world data, particularly in sensitive domains like healthcare and well-being. This data scarcity can hinder a robot's ability to adapt to new situations. To address these challenges, causality provides a structured framework for understanding and modeling the underlying relationships between actions, events, and outcomes. By moving beyond mere pattern recognition, causality enables robots to make more explainable and generalizable decisions. This paper presents an exploratory causality-based analysis through a case study of an adaptive robotic coach delivering positive psychology exercises over four weeks in a workplace setting. The robotic coach autonomously adapts to multimodal human behaviors, such as facial valence and speech duration. By conducting both macro- and micro-level causal analyses, this study aims to gain deeper insights into how adaptability can enhance well-being during interactions. Ultimately, this research seeks to advance our understanding of how causality can help overcome challenges in HRI, particularly in real-world applications.
BibTeX
@inproceedings{spitale2025exploring,
  author = {Micol Spitale and Srikar Babu and Serhan Cakmak and Jiaee Cheong and Hatice Gunes},
  title = {{Exploring Causality for HRI: A Case Study on Robotic Mental Well-being Coaching}},
  booktitle = {RO-MAN},
  year = {2025},
  doi = {10.1109/ro-man63969.2025.11217880},
}
Figure