Human-Aware AI - A Foundational Framework for Human-AI Interaction

Sarath Sreedharan

Introduction

Despite the rapid advancement of artificial intelligence, a significant gap remains in how these systems interact with human users. This disconnect is often characterized as a "double empathy problem"—a term borrowed from autism research to describe the fundamental mismatch between two different "operating systems." In the context of AI, the problem is not necessarily a lack of social intelligence, but rather that AI represents a "truly alien intelligence." Its internal logic often diverges significantly from the mental models humans construct to understand it. When an AI's actions contradict human expectations, trust erodes, and coordination breaks down. Current AI systems frequently lack explainability and controllability because they ignore the human's perspective in the loop.

The Human-Aware AI (HAAI) Framework

The Human-Aware AI (HAAI) framework proposes a foundational solution to this disconnect by centering on mental modeling. In a standard interaction, an AI agent operates based on its own internal model ($M^R$). However, the HAAI framework posits that the system must also account for the human’s belief about the agent's model ($M^R_h$).

The Human-Aware AI framework models the gap between the agent's actual model ($M^R$) and the human's belief about that model ($M^R_h$).

The primary goal of HAAI is to bridge the "expectation mismatch" between the agent's actual behavior and the behavior the human expects. By explicitly modeling $M^R_h$, the AI can generate behaviors or explanations that align with human intuition, even when the underlying computational logic is complex or non-intuitive.

To effectively bridge this gap, the framework identifies three critical dimensions of asymmetry that AI systems must navigate:

  1. Asymmetry in Knowledge: The AI and the human often have access to different sets of information regarding the environment or the task at hand.
  2. Asymmetry in Inferential Capabilities: Humans and AI process information using different mechanisms. While an AI might perform an exhaustive search of a state space, a human might rely on heuristics or qualitative reasoning.
  3. Asymmetry in Vocabulary: The internal representations and "language" used by the AI to make decisions often do not map directly to human concepts, necessitating a translation layer.

Key Insights and Applications

The HAAI framework shifts the benchmark for AI performance from mere accuracy to expectation alignment. Coordination is achieved when the AI's actions are predictable and understandable within the context of the human's mental model. This goes beyond simple interpretability; a "human-aware" system actively manages the three asymmetries to ensure the human understands why a decision was made.

The framework has already shown promise in several areas:

  • Explanation Generation: Developing methods that specifically target and resolve knowledge or vocabulary gaps.
  • Human-Advising: Moving toward systems that can effectively receive and integrate guidance from human experts by understanding the context of that advice.
  • Value Alignment: Ensuring that as AI systems grow in complexity, their objective functions remain aligned with human ethics and intentions by accounting for the human's mental model of the task.

Conclusion and Future Directions

While the HAAI framework provides a robust conceptual roadmap, the next frontier involves moving from theoretical modeling to practical, algorithmic implementations. Future work will focus on refining how AI systems integrate human advice and ensuring long-term value alignment in increasingly autonomous systems. By treating human-AI interaction as a two-way street of mutual understanding, HAAI offers a path toward more trustworthy and collaborative artificial intelligence.

Summary updated

Abstract

We are living through a revolutionary moment in AI history. We are seeing the development of impressive new AI systems at a rate that was unimaginable just a few years ago. However, AI's true potential to transform society remains unrealized, in no small part due to the inability of current systems to work effectively with people. A major hurdle to achieving such coordination is the inherent asymmetry between the AI system and its users. In this talk, I will discuss how the framework of Human-Aware AI (HAAI) provides us with the tools required to bridge this gap and support fluent and intuitive coordination between the AI system and its users.
BibTeX
@inproceedings{sreedharan2023human,
  author = {Sarath Sreedharan},
  title = {{Human-Aware AI - A Foundational Framework for Human-AI Interaction}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2023},
  doi = {10.1609/aaai.v37i13.26822},
}
Figure