User Modelling

Derek H. Sleeman, Douglas E. Appelt, Kurt Konolige, Elaine Rich, N. S. Sridharan, William R. Swartout

Introduction

In the mid-1980s, as Artificial Intelligence began to move from laboratory curiosities to practical applications, a critical question emerged: how can a system understand its user? This summary captures the essence of a seminal panel discussion from IJCAI 1985, moderated by Derek Sleeman. The panel explored the emerging field of User Modelling—the development of AI systems that maintain internal representations of human users to provide tailored, context-aware responses.

The Problem: Beyond One-Size-Fits-All

Early AI systems, whether designed for tutoring or expert diagnosis, typically treated every user as an identical entity. This rigidity limited their effectiveness in complex domains like education or natural language dialogue. The panel identified three core challenges:

  1. Inference: How can a system deduce a user's knowledge or goals from limited behavioral observations?
  2. Representation: What is the optimal structure for a user model? Should it be a list of facts, a network of beliefs, or a generalized "stereotype"?
  3. Utility: Is the computational overhead of a high-fidelity model justified by the improvement in system performance?

Conceptual Frameworks

The panel synthesized several key approaches to structuring user models, most notably the classification of information types.

The 2x2 Classification Matrix

Elaine Rich proposed a foundational framework for categorizing user information along two axes: the specificity of the user and the duration of the information.

Figure 1: Elaine Rich's 2x2 matrix for classifying user information based on specificity (individual vs. canonical) and duration (short-term vs. long-term).

This matrix distinguishes between "canonical" models (generalized classes of users) and "individual" models (specific personas), as well as "short-term" session goals versus "long-term" permanent knowledge.

Tutoring and Mental Models

D. Sleeman introduced the distinction between Overlay models—where a student's knowledge is viewed as a subset of an expert's—and Perturbation models. The latter explicitly accounts for "bugs" or incorrect reasoning patterns, allowing the system to address misconceptions rather than just missing facts.

Rational Agency and Plan Recognition

Other contributors looked toward formal logic and planning. Kurt Konolige applied the Belief-Desire-Intention (BDI) paradigm, treating the user as a rational agent. N.S. Sridharan and Doug Appelt argued that user modelling is essentially a task of Plan Recognition: the system must hypothesize the user's underlying plan in real-time to provide proactive assistance.

Key Insights

  • Inductive Inference: Unlike standard deductive AI, user modelling is inherently inductive. Systems must "induce" hidden mental states from observable behavior, a process that is fundamentally error-prone.
  • The Role of Stereotypes: Bill Swartout suggested that AI should mimic human communication by relying on "stereotypes" (generalized models) and refining them only when feedback indicates a discrepancy.
  • Communication as Intention: True communication requires a model of what the user intends for the system to believe. Without intention recognition, dialogue remains superficial.

Conclusion

The panel concluded that the complexity of a user model should be proportional to the "richness of the response space." If a system only has a few possible actions, a complex model is unnecessary. However, as AI systems grow more sophisticated, the responsibility for modelling must shift from the user (the "Glass Box" approach) to the system (the "Black Box" approach) to prevent cognitive overload. The discussion established plan recognition and the management of evolving user models as fundamental, yet computationally daunting, challenges for the field.

Summary updated

BibTeX
@inproceedings{sleeman1985user,
  author = {Derek H. Sleeman and Douglas E. Appelt and Kurt Konolige and Elaine Rich and N. S. Sridharan and William R. Swartout},
  title = {{User Modelling}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {1985},
}
Figure