A Database For A.I.

J. Quinton, A. AndrewIJCAI 1975

Abstract: This paper explores the "Cognitive Test-Bed" (CTB), a late-1960s system designed to record high-resolution data on human problem-solving during man-machine interaction. By having subjects play the game "Peggity" (a five-in-a-line variant) against a computer, researchers aimed to build a database of cognitive strategies—specifically the acquisition of "Keys" (winning patterns)—to better inform the development of learning programs in AI.

In the late 1960s and early 1970s, a pivotal era for artificial intelligence, researchers J. G. St. Quinton and A. M. Andrew sought to bridge the gap between human psychology and machine learning. Their work centered on "The Cognitive Test-Bed" (CTB), a system designed to capture high-resolution data on human problem-solving during man-machine interactions.

The Cognitive Test-Bed (CTB)

The primary challenge addressed by the CTB was the lack of high-quality reference data on human information processing. Traditional methods, such as "verbal protocols" where subjects describe their thoughts aloud, were often intrusive and lacked the precision required for computational modeling. The CTB offered a "control" environment where human cognitive activity could be monitored in real-time without interrupting the task's flow, creating a dataset where human learning could be measured against a known algorithmic opponent.

Methodology and Data Analysis

The researchers implemented a two-stage system for data handling:

  1. Data Acquisition: Using a DEC PDP 12 computer, 40 graduate students played Peggity. The system recorded moves via joystick, displayed the board on a CRT, and meticulously measured "move latency"—the time taken for a human to decide on a move.
  2. Data Analysis: An ICL 1904 S computer processed the raw data using three specialized programs: SCAN-TEST (evaluating move outcomes), KEY DESCRIPTOR (identifying winning patterns), and HEURISTIC PROBE (searching for strategies like backtracking or means-ends analysis).

To ensure a consistent challenge, the computer opponent operated at three strategy levels that scaled with the human player's performance.

Figure 2: A board lattice line slice used by the CTB to analyze patterns around a specific node 'a'.

The "Key" Metaphor and TG Pairs

The authors conceptualized problem-solving as navigating a building with locked doors. To reach a "Goal State," a player must acquire "Keys"—specific information or board patterns—and "Heuristics" to utilize them. In Peggity, these keys are patterns like "crossed threes" that are essential for winning.

To quantify these patterns, the system used "TG pairs" (Target-Goal pairs), which served as board position vectors. These pairs allowed the researchers to mathematically represent the state of the game and the potential value of any given move, providing approximately 10 informational items per move in the final database.

Figure 4 (iv): An example of a TG pair (2,3) associated with a specific board configuration.

Key Insights and Results

The study yielded several fascinating observations about human cognition:

  • Perceptual Bias: Humans exhibited a significant bias toward horizontal and vertical lines. Subjects were much more likely to overlook threats or opportunities on diagonal axes, a finding that highlights the limitations of human spatial perception in strategic contexts.
  • The "Turing" Effect: The computer's behavior was perceived as remarkably "life-like." Despite the relative simplicity of the algorithms, several subjects were convinced they were playing against a hidden human opponent.
  • AI as a Scientific Control: The researchers validated the machine as a stable control. While the machine's evaluative functions remained constant, human learning and strategy evolved, allowing for a clear measurement of cognitive progression.

On average, human subjects won only one in seven games. However, the data showed clear progression; while 50% of female subjects reached Strategy Level 2, 20% of male subjects successfully reached the highest level (Level 3).

Conclusion and Future Directions

While the "Keys" identified were specific to Peggity, the CTB demonstrated the potential for creating robust databases of human heuristic processes. The authors suggested that future research should expand the subject pool to explore how cognitive processing varies across different demographics and personality types. This historical work remains a testament to the early efforts to ground artificial intelligence in the empirical study of human intelligence.

Reference: A Database For A.I. by J. G. St. Quinton and A. M. Andrew, University of Reading.

Summary updated

BibTeX
@inproceedings{quinton1975database,
  author = {J. Quinton and A. Andrew},
  title = {{A Database For A.I.}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {1975},
}
Figure