Abstract
Raven’s Progressive Matrices (RPMs) are a cornerstone of non-verbal intelligence testing, measuring general fluid intelligence ($g$) through geometric pattern completion. Traditionally, these problems require expert manual design, limiting their availability for large-scale or adaptive testing. This paper introduces a formal framework based on First-Order Logic (FOL) to automatically synthesize high-quality RPMs. Our system generates hundreds of "well-formed" problems per second that are statistically indistinguishable from human-designed benchmarks in user studies.
Introduction
Raven’s Progressive Matrices (RPMs) are widely regarded as the gold standard for cross-cultural intelligence assessment because they minimize linguistic and cultural bias. A typical RPM presents a 3x3 grid of geometric patterns with one missing cell, requiring the test-taker to identify the underlying logical rules to select the correct completion from a set of distractors.
Despite their utility, the scarcity of high-quality RPMs is a significant hurdle. Manual creation is labor-intensive, and simple template-based generation often results in predictable or logically flawed problems. To address this, we propose a logic-based synthesis engine that treats RPM generation as a constraint satisfaction problem.
Formalizing the Logic of RPMs
The core of our approach is the translation of informal geometric patterns into First-Order Logic (FOL). We model RPMs as collections of objects defined by attributes such as shape, size, and orientation. We categorize the logical relationships into three fundamental types:
- Unary Relations: A single element determines the next in a sequence (e.g., progressive rotation or scaling).
- Binary Relations: Two elements combine to determine the third (e.g., $A + B = C$ via logical operations like XOR).
- Ternary Relations: A constant constraint applies across all three elements in a row or column (e.g., every row must contain exactly one circle, one square, and one triangle).
By decoupling this abstract logic from the visual rendering, our system can generate a vast diversity of problems from a single logical structure by varying the geometric primitives used.
Ensuring Well-Formedness
A critical challenge in automatic generation is ensuring that problems are "well-formed"—meaning they are solvable, unambiguous, and challenging. We define three formal properties for a valid RPM:
- Prerequisite: The matrix must contain at least one satisfiable logical rule.
- Correctness: There must be exactly one answer among the choices that satisfies all rules.
- Necessity: Every rule included in the problem must be necessary to reach the unique solution. If a solver ignores one rule, they should find multiple distractors that appear plausible, preventing success through partial reasoning or guessing.
Our synthesis engine uses an optimization process based on "Maximal Proper Subsets" of constraints to generate the seven incorrect distractor choices, ensuring they effectively test the subject's ability to synthesize all relevant information.
Evaluation and Results
We evaluated our system by synthesizing a corpus of 200 RPMs and comparing them against the Advanced Progressive Matrices (APM), the standard version for adults. In a user study involving 24 participants (ranging from students to faculty), we found that error rates on synthesized problems were statistically indistinguishable from those on official APM problems.
Furthermore, the system is highly efficient, generating a complete, well-formed RPM in less than 4 milliseconds on average. This performance enables real-time generation for personalized or adaptive testing environments. We also confirmed that difficulty can be precisely controlled by adjusting the number of logical constraints or adding "noise" attributes that do not follow any rule.
Conclusion
By formalizing the underlying logic of geometric reasoning, we have developed a system capable of producing an inexhaustible supply of high-quality IQ test problems. This work bridges the gap between symbolic logic and psychometric assessment, offering a scalable solution for educational and clinical testing. Future work will focus on expanding the library of visual transformations to include complex Gestalt-based patterns.
Summary updated