Towards Human-Engaged AI

Xiaojuan MaIJCAI 2018 · DOI

TL;DR

This paper introduces "Human-Engaged AI" (HEAI), a paradigm focused on creating synergistic interactions between humans and AI. It proposes a framework for understanding and fostering engagement, moving beyond mere AI capability to a more holistic human-AI partnership.

What Problem Does This Paper Address?

The paper addresses the growing need for AI systems that don't just perform tasks but actively engage with human users. While AI and Human-Computer Interaction (HCI) fields are converging, current AI often overlooks the crucial aspect of "engagement"—the synergistic relationship and mutual enhancement between humans and AI. This can lead to AI systems that are technically capable but fail to integrate effectively or provide a truly fulfilling user experience. The authors aim to bridge this gap by defining and operationalizing HEAI.

Methods

The paper outlines a data-driven HEAI framework with five key components:

  1. Computational Model of Human Engagement: Developing models to understand how humans engage with AI.
  2. Real-time Inference of Human Engagement: Using various signals to infer a user's engagement status in real-time.
  3. Management of Human Engagement: Strategies for AI to influence and guide user engagement.
  4. Expression of AI Engagement: How AI systems can communicate their own engagement status to users.
  5. Engagement-based Applications: Utilizing HEAI principles in practical applications.

The authors emphasize using a combination of social signals (gaze, facial expressions, vocal behaviors, posture), physiological signals (ECG, pulse, skin conductance), and behavioral signals (interface logs, actions, reactions) to infer engagement. They also propose two mechanisms for engagement management: user initiative (evoking user awareness through data visualization) and system initiative (AI proactively intervening, e.g., using proxemics or personality theory).

Datasets

Not explicitly discussed in the paper. The authors refer to their own past research and cite numerous works that likely utilize various datasets for training and evaluation of engagement models and AI systems.

Key Insights and Ideas

  • Engagement as a Core Construct: The paper elevates "engagement" from a desirable user experience trait to a fundamental construct for effective human-AI collaboration.
  • Mutual Enhancement: HEAI aims for a symbiotic relationship where both humans and AI benefit and enhance each other's capabilities.
  • Beyond Usability: HEAI moves beyond simply making AI "usable" to creating interactions that are "fulfilling, satisfying, and creative."
  • Multi-modal Engagement Sensing: The paper highlights the importance of combining social, physiological, and behavioral signals for a comprehensive understanding of human engagement.
  • AI Needs Social and Emotional Intelligence: To achieve HEAI, AI systems need to be equipped with capabilities akin to human social and emotional intelligence to manage interactions effectively.
  • Two-Way Street: Both humans and AI need to be "engaged" and "engaging" for optimal synergy.

Key Results

The paper presents a conceptual framework and outlines research directions rather than reporting specific quantitative results from a single study. The "results" are more in the form of demonstrated capabilities and research explorations:

  • Demonstrated Engagement Sensing: The authors have explored using sensors (cameras, depth sensors, eye trackers) to capture social signals and physiological signals (skin conductance) to infer engagement.
  • Engagement Management Strategies: They have investigated user-initiative approaches (e.g., data visualization of engagement) and system-initiative approaches (e.g., using proxemics for posture adjustment, personality theory for virtual agents).
  • AI Engagement Expression: Research into how AI can express its own engagement through non-verbal cues (facial expressions, movement, voice waveform) to improve user connection.
  • Application Examples: HEAI principles have been applied in areas like online learning, digital games, social media analysis, and voice assistants.

Limitations and Future Work

Limitations: * The paper focuses on a conceptual framework and research landscape, with specific quantitative results from a single study not being the primary focus. * The complexity of inferring and managing engagement across diverse contexts and user groups remains a significant challenge.

Future Work: * Developing more robust and generalizable computational models for human engagement. * Exploring novel methods for real-time, non-intrusive engagement inference. * Designing more sophisticated AI engagement expression strategies. * Investigating the long-term impact of HEAI on user experience and task performance. * Expanding HEAI applications to a wider range of domains and AI technologies.

Summary updated

BibTeX
@inproceedings{ma2018towards,
  author = {Xiaojuan Ma},
  title = {{Towards Human-Engaged AI}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2018},
  doi = {10.24963/ijcai.2018/809},
}
Figure