Can a System Be Intelligent if It Never Gives a Damn?

Thomas EdelsonAAAI 1986

Abstract

A central debate in artificial intelligence concerns whether true intelligence requires "affect"—the capacity to care, feel emotion, or have a stake in the world. This paper explores the specific challenge of social intelligence: the ability to reason about human behavior. Thomas Edelson argues that while a system must possess a sophisticated model of human emotions to understand us, it does not need to experience those emotions itself. Using a "virtual machine" analogy, Edelson demonstrates how we can decouple the knowledge of emotion from the experience of it, leading to AI that is socially competent yet remains a predictable, indifferent tool.

Introduction: The Necessity of Caring

In the mid-1980s, philosophers like John Haugeland argued that "pure" cognition was a myth. To Haugeland, a truly intelligent system must be an "engaged system"—one that has something at stake in its environment. Without the ability to "give a damn," it was argued, a system could never truly understand the world or the humans within it.

Edelson addresses this challenge by focusing on social intelligence. If humans rely on their own emotional hardware to understand others, can a computer—which lacks that biological hardware—ever achieve human-level performance in social reasoning? The paper explores whether we can decouple the internal mechanisms used to model emotion from the subjective experience of feeling them.

The Virtual Person Hypothesis

Edelson proposes that human social understanding is rooted in a "Virtual Person" hypothesis. We understand others by "imagining ourselves into their situation," effectively providing "fake" sensory inputs to our own affective brain processes to observe the resulting emotional output.

To translate this into computational terms, Edelson employs the analogy of IBM’s VM/370 architecture. In this framework, a host computer runs a "virtual" guest machine by intercepting its input/output operations. Similarly, Edelson suggests a human "runs" a simulation of another person on their own biological hardware. For AI, this implies that the most efficient way to model a complex human system is to use a similar internal model as a template. This "simulation as heuristic" approach suggests that while we could try to model human behavior as a "black box," it is far more effective to build a model based on the internal mechanisms of how affect actually works.

Flo vs. Fred: Decoupling Affect from Cognition

To illustrate that understanding does not require personal engagement, Edelson introduces a thought experiment involving two hypothetical systems:

  • Fred: An "engaged" system. Fred possesses his own motives and desires. If a user insults Fred, he gets angry and may refuse to cooperate.
  • Flo: An "unengaged" system. Flo contains an internal model of a human (named "Moe"). When insulted, Flo uses the "Moe" model to predict how a human would react, concluding that the user is being hostile. However, Flo herself remains indifferent and continues her professional duties.

Edelson argues that Flo possesses a functional understanding of emotion. Against critics like Searle or Haugeland, he posits that if a system consistently provides correct answers about human emotions, it understands those emotions, regardless of whether it "feels" them. This suggests that affective faculties can be treated as a module—internalized for reasoning purposes without being granted control over the system's global behavior.

Conclusion: The Case for Indifferent AI

Edelson’s analysis leads to three primary conclusions. First, social intelligence is likely interdependent with affective modeling; we cannot build systems that understand human-level concepts without first figuring out how to model "caring." Second, building a system that understands caring does not necessitate building a system that actually cares.

Finally, for practical applications, "unengaged" systems like Flo are preferable. In contexts like medical expert systems, a user requires an AI that understands their fear and pain but does not become personally offended or emotionally volatile. While implementing such a "decoupled" system remains a significant challenge, it offers a path toward AI that is both deeply intelligent and reliably professional.

Summary updated

BibTeX
@inproceedings{edelson1986can,
  author = {Thomas Edelson},
  title = {{Can a System Be Intelligent if It Never Gives a Damn?}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {1986},
}

Loading related content...

Figure