An Analysis of Psychological Experiments on Non-Monotonic Reasoning

Edward HoenkampIJCAI 1987

Abstract

This paper bridges the gap between Artificial Intelligence and psychology by using AI formalisms—specifically Truth Maintenance Systems (TMS)—to model why humans often fail to retract beliefs even after the evidence for them has been completely discredited. By formalizing "irrational" belief perseverance as a logical system where self-generated justifications take on a life of their own, the research provides a descriptive model for human cognitive biases.

Introduction: The Challenge of Non-Monotonic Reasoning

In Artificial Intelligence, "non-monotonic reasoning" refers to the ability to retract a conclusion when new, contradictory information arises. While AI researchers often look to human common sense as a gold standard for such reasoning, psychological evidence suggests a significant discrepancy. "Dehoaxing" experiments reveal that humans frequently exhibit a "perseverance effect": even when the initial basis for a belief is revealed to be entirely fabricated, the belief often persists. This paper addresses this gap by proposing a formal model that accounts for this seemingly irrational behavior.

Empirical Basis: Psychological Studies of Perseverance

The paper performs a meta-analysis of well-documented psychological experiments to establish the ubiquity of belief perseverance. The primary benchmark is the study by Ross, Lepper, and Hubbard (1975), which focused on subjects' self-perceptions of performance after receiving false feedback.

Summary of psychological studies demonstrating belief perseverance across various domains.

As shown in the table above, the phenomenon spans diverse domains, including social aptitude, physiological feedback, and even political preferences. These studies consistently show that debriefing information (discrediting the initial deception) often fails to eliminate the induced belief.

Methods: Modeling Belief with Truth Maintenance Systems

The author employs Doyle’s Truth Maintenance System (TMS) to model these cognitive states. In a TMS, beliefs are represented as nodes linked by justifications. A belief is considered "IN" (accepted) if it has a valid justification and "OUT" (rejected) otherwise.

The core of the model is the inclusion of "self-generated confirmatory evidence." When a subject receives feedback (e.g., "you are good at this"), they don't just store the statement; they search their memory for supporting evidence (e.g., "I did well on a similar task once"). This creates an independent justification node in the mental TMS that survives even if the original feedback node is discredited.

Key Results and Formalization

The paper successfully translates the Ross et al. experiment into a formal set of logical reasons. By computing the "deductive closure" of these reasons, the model generates distinct belief states or "extensions."

Formalization of belief states and dependencies using Truth Maintenance System (TMS) logic.

The table above illustrates how beliefs (a through e) interact. Even when the external evidence (c) is removed or discredited (d), the self-generated evidence (e) can keep the primary belief (a) "IN." This formalization explains why "outcome debriefing" (simply stating the data was fake) is less effective than "process debriefing," which explicitly targets the psychological mechanism of self-justification.

Insights and Implications

  • The Independent Support Trap: Perseverance occurs because the original "trigger" evidence is no longer the sole support for the belief. A scaffold of self-generated reasons maintains the belief's validity within the subject's internal logic.
  • Predictive Power: The model predicts that perseverance can occur even if subjects are warned beforehand that feedback will be bogus. This counter-intuitive prediction aligns with subsequent psychological findings (e.g., Wegner et al., 1985).
  • AI as a Descriptive Tool: The results demonstrate that TMS can serve as a powerful descriptive tool for human psychology, moving beyond its traditional role as a prescriptive tool for AI logic.

Conclusion

While the model is relatively simple, it provides a rigorous framework for understanding human irrationality. Future work could extend this logic to more complex social scenarios, such as the persistence of political misinformation or the impact of discredited testimony in legal settings.

Summary updated

BibTeX
@inproceedings{hoenkamp1987analysis,
  author = {Edward Hoenkamp},
  title = {{An Analysis of Psychological Experiments on Non-Monotonic Reasoning}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {1987},
}
Figure