From the Programmer's Apprentice to Human-Robot Interaction: Thirty Years of Research on Human-Computer Collaboration

Charles Rich, Candace L. SidnerAAAI 2006

Introduction

Presented at AAAI 2006, this retrospective by Charles Rich and Candace Sidner chronicles three decades of research aimed at transforming computers from mere tools into true collaborators. Their work traces an evolution from specialized software engineering assistants to a generalized theory of human-computer collaboration, ultimately extending into the realm of human-robot interaction (HRI).

Theoretical Foundations: Modeling Human Collaboration

The core philosophy of this research thread is the "substitution principle": a computer agent should ideally be able to replace one human in a two-person team without altering the fundamental structure of the interaction. To achieve this, the authors grounded their work in the principles of human-human collaboration.

Conceptual model of human-computer collaboration based on human-human interaction.

Central to this model is SharedPlan Theory, developed in conjunction with Barbara Grosz. This theory formalizes how agents coordinate their actions and beliefs to achieve common goals. Collaboration is viewed as a tripartite discourse process involving:

  • Intentionality: Maintaining mutual beliefs and shared intentions regarding the task (SharedPlans).
  • Attention: Tracking the "focus of attention" to manage what is currently being discussed.
  • Linguistics: Handling the actual sequence of utterances and communicative acts.

Engineering Collaboration: The Collagen Framework

To move from theory to practice, the authors developed Collagen (Collaborative Agent), a middleware framework designed to simplify the creation of collaborative systems across diverse domains.

The architecture of the COLLAGEN middleware framework.

As shown in the architecture, Collagen acts as a bridge between the user, the application, and the agent. It utilizes a "Recipe Library" to store task models and a "Discourse State" to track the progress of the SharedPlan. By managing the focus stack and interpreting user utterances or actions, Collagen allows developers to focus on domain-specific logic while the middleware handles the complexities of collaborative discourse.

Evolution and Application Domains

The research journey began with "The Programmer’s Apprentice," an ambitious attempt to build a collaborator for software engineers. However, the authors soon realized that the sheer volume of domain-specific knowledge required made the task intractable. This led to a pivot toward application-independent tools.

Over the years, the theories and the Collagen framework were validated across numerous testbeds, including:

  • Air Travel and Email: Early prototypes for personal assistance.
  • Industrial Training: Systems like PACO for gas-turbine operation.
  • Consumer Electronics: The "DiamondHelp" interface, which used a chat-based metaphor to help users program complex home appliances like thermostats and VCRs.
  • Human-Robot Interaction: Recent work focused on "engagement"—the nonverbal cues (gaze, nodding) necessary for physically embodied robots to maintain a connection with human partners.

Key Insights

  • Collaboration and Discourse are Inseparable: The authors argue that effective collaboration is fundamentally a discourse process. You cannot have one effectively without the other.
  • Contextual Tractability: While general plan recognition is computationally difficult (NP-complete), the constraints of a collaborative setting—specifically the shared focus of attention and hierarchical plans—make it tractable in real-time.
  • The Knowledge Bottleneck: Despite the success of Collagen, the need for a manually codified "Recipe Library" for each new domain remains a significant challenge for scaling collaborative AI.

Conclusion

Rich and Sidner’s thirty-year odyssey provides a rigorous logical and engineering foundation for human-computer collaboration. By treating the computer as a peer rather than a tool, their work paved the way for modern intelligent assistants and social robots, emphasizing that true intelligence lies not just in solving problems, but in working together to solve them.

Summary updated

BibTeX
@inproceedings{rich2006from,
  author = {Charles Rich and Candace L. Sidner},
  title = {{From the Programmer's Apprentice to Human-Robot Interaction: Thirty Years of Research on Human-Computer Collaboration}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2006},
}
Figure