Learning about Representational Modality: Design and Programming Projects for Knowledge-Based AI

AAAI 2013

TL;DR

This paper describes a novel approach to teaching knowledge-based AI by using a series of programming projects based on the Raven's Progressive Matrices intelligence test. The projects successfully engaged students and taught them about representational modality, a concept often overlooked in traditional AI curricula.

What Problem Does This Paper Address?

Traditional AI courses often focus on propositional representations, neglecting the crucial aspect of representational modality (e.g., propositional vs. iconic). This oversight can limit students' understanding of how different representation types impact AI system design and performance. The authors aimed to address this gap by designing a curriculum that explicitly teaches students about representational modality through hands-on programming projects. They chose the Raven's Progressive Matrices (RPM) test as the domain because it's a well-established measure of intelligence that involves complex reasoning and has been shown to involve different representational strategies.

Methods

The core of the paper is the description of a four-project sequence designed for a knowledge-based AI (KBAI) course. The authors employed an "action research" methodology, where the teacher (also a researcher) iteratively designs, implements, and reflects on interventions in the classroom to improve learning.

The projects progressively built upon each other:

  • Project 1: Focused on solving 2x1 analogy problems using propositional representations. Students were given visual problems and had to create their own propositional representations and a program to solve them.
  • Project 2: Scaled up to 2x2 and 3x3 matrix problems, still primarily using propositional representations.
  • Project 3: Shifted to direct manipulation of input images, encouraging students to explore iconic representations. This project introduced students to the idea of working directly with visual data.
  • Project 4: Required students to implement both propositional and visual reasoning strategies within the same program, enabling a direct comparison of their effectiveness.

Datasets

The "datasets" used in this study were derived from the Raven's Progressive Matrices (RPM) test and the Miller analogy test. The authors provided students with visual representations of these problems. For Project 1, they also provided sample propositional representations to guide students. The problems themselves served as the benchmark for evaluating student programs.

Key Insights and Ideas

  • Engaging Students with Abstract Concepts: The RPM test proved to be a highly motivating and engaging domain for teaching complex AI concepts, leading students to invest significant time and effort.
  • Bridging the Modality Gap: The project sequence effectively guided students from understanding propositional representations to exploring and comparing them with iconic representations, highlighting the importance of representational modality.
  • Action Research in Education: The iterative design and reflection process of action research allowed the instructors to refine the projects based on student performance and feedback, leading to a more effective learning experience.
  • Student Ownership and Introspection: Students reported feeling like "real computer scientists" and were prompted to introspect about their own intelligence and problem-solving strategies.

Key Results

  • High Student Engagement: Students found the projects motivating, engaging, and challenging, with many going beyond the basic requirements and posting their work online.
  • High Performance: Median scores for undergraduate students in Fall 2012 were very high (99, 96, 95, 85 for the four projects), indicating successful learning outcomes. Graduate student scores were even higher.
  • Qualitative Feedback: Students provided overwhelmingly positive feedback, appreciating the research-level nature of the projects, the freedom to choose languages, and the opportunity to think about AI in new ways.
  • Identified Mismatch: A key qualitative finding was that some students felt the projects didn't directly align with the traditional propositional focus of the rest of the course. This feedback informed future course design to make connections more explicit.

Limitations and Future Work

  • Perceived Disconnect: Some students felt the projects, while interesting, did not directly relate to the traditional propositional AI topics covered in the rest of the course. The authors acknowledge this and suggest future work to explicitly bridge this gap by emphasizing representational modality alongside traditional topics.
  • Time Commitment: A few students found the projects to be very time-consuming, suggesting that perhaps fewer projects or a different assignment structure might be considered.
  • Future Iterations: The authors plan to address student concerns about making connections between the projects and the rest of the course more explicit in future iterations. They also aim to further refine the projects and problems based on ongoing observations.

Summary updated

BibTeX
@inproceedings{2013learning,
  title = {{Learning about Representational Modality: Design and Programming Projects for Knowledge-Based AI}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2013},
}
Figure