Abstract
Expertise in problem-solving is often characterized by the strategic use of analogy. This study investigates the specific mechanisms that differentiate expert-level analogical reasoning from novice approaches. By systematically "ablating" strategic features in an AI thermodynamics solver, we demonstrate that superior performance is a product of rigorous problem encoding, validation of analogical inferences against first principles, and the consideration of multiple potential analogies.
Introduction
The role of analogy in expertise has historically been a point of contention. Some researchers argued that experts rely primarily on first-principles reasoning, while others viewed analogy as a fundamental cognitive tool across all skill levels. This paper proposes that the distinction lies not in the presence of analogy, but in its execution. The authors aim to identify the strategic variations—such as problem description depth and solution verification—that separate efficient, expert-like problem solving from the error-prone methods typical of novices.
Methodology
The study utilizes the Analogical Problem Solving System (APSS), which integrates two core cognitive simulations: the Structure-Mapping Engine (SME) for analogical alignment and MAC/FAC for case retrieval.
The Expert Model
The "expert" strategy is defined by three primary characteristics:
- Rigorous Encoding: Problem descriptions include qualitative physical laws and relationships (e.g., identifying a "piston-cylinder device in equilibrium") rather than just numerical values.
- Multiple Retrievals: The system examines the top three similar cases retrieved from memory, rather than relying solely on the single best match.
- Validation: Analogical suggestions are rigorously checked against first-principles knowledge before being implemented.
Experimental Design
The system was evaluated using a corpus of 82 textbook thermodynamics problems. Using a "leave-one-out" methodology, the system attempted to solve each problem by drawing upon the remaining 81 problems as its case library. Performance was measured by the number of nodes (search steps) expanded and the accuracy of the final solution.
Key Insights: Analogy as an Efficiency Booster
A critical finding is that analogy serves as a powerful tool for managing search space complexity. While first-principles reasoning is sufficient for simple tasks, its efficiency degrades as problem complexity increases. Analogy mitigates this by providing a targeted path through the search space.
Furthermore, the study highlights the importance of Qualitative Encoding. By representing conceptual relationships, the retrieval mechanism can identify analogies based on structural similarity rather than surface-level keywords. Validation also proved essential; skipping verification steps often led to "dead-end" paths, ultimately increasing computational effort and leading to errors.
Results and Discussion
The "Expert" model consistently outperformed all ablated variations in both efficiency and accuracy.
As shown in the results, the Expert model averaged 20.2 nodes per problem with zero errors. In contrast:
- Simple Encoding resulted in a ~9% performance decrease (21.9 nodes), with the disparity most pronounced in complex problems.
- Single Reminding led to a ~14% performance hit (23.0 nodes), demonstrating the value of redundancy.
- No Validation models, while appearing slightly faster in some instances (19.8 nodes), were prone to errors and became significantly less efficient when faced with complex, misleading analogies.
Conclusion
This research clarifies the strategic nuances of expert problem-solving. It demonstrates that analogy is not a shortcut that bypasses rigor, but a sophisticated process that requires deep encoding and constant validation. Future work will explore whether these strategies generalize to non-mathematical domains and integrate these findings into "Companion Cognitive Systems" to enhance human-AI collaboration.
Summary updated