MP: Endowing Large Language Models with Lateral Thinking

Tian Bai, Yongwang Cao, Yan Ge, Haitao YuAAAI 2025 · DOI

Abstract

While Large Language Models (LLMs) have demonstrated exceptional proficiency in "vertical thinking"—the logical, step-by-step reasoning required for mathematics and programming—they frequently falter when confronted with "lateral thinking" tasks. Lateral thinking requires identifying indirect clues, puns, and metaphors where the obvious logical path is often a distractor. This paper introduces Metacognitive Prompting (MP), a novel three-step prompting strategy inspired by human metacognition. By structuring the reasoning process into planning, initial reasoning, and deep reflection, MP enables LLMs to navigate complex linguistic traps. Our evaluations show that MP significantly outperforms standard prompting techniques, with GPT-4 even surpassing human benchmarks on the BRAINTEASER dataset.

Introduction

The primary challenge in lateral thinking for AI is the lack of metacognitive awareness—the ability to monitor, regulate, and correct one's own creative reasoning processes. Standard techniques like Chain-of-Thought (CoT) help models decompose problems but do not inherently equip them to recognize when they are being "tricked" by surface-level information. For instance, a model might fail to see that a question about the "capital in London" refers to the letter 'L' rather than a geographical entity. MP addresses this gap by formalizing a "thinking about thinking" loop.

The Metacognitive Prompting Framework

The authors propose a model-agnostic framework that structures model interaction into three distinct phases, mirroring human cognitive regulation.

Figure 1: Comparison of human metacognition and the proposed metacognitive prompting framework for LLMs.
  1. Planning (Strategy Formulation): The model first identifies the task type and selects an appropriate strategy based on "cognitive knowledge." It recognizes that brain teasers often involve pitfalls like double entendres or misleading distractors.
  2. Reasoning (Initial Analysis): Guided by the chosen strategy, the model identifies specific wordplay or implicit information and generates an initial answer and explanation.
  3. Reflecting (Deep Assessment): The model evaluates its own reasoning. It checks if the initial logic holds up or if it was misled by surface-level cues. This self-correction loop is vital for catching "hallucinations of logic."

Experimental Evaluation

The researchers evaluated MP across three major benchmarks designed to test creative problem-solving:

  • BRAINTEASER: Features "Sentence Puzzles" (situational riddles) and "Word Puzzles" (linguistic tricks).
  • BiRdQA: A bilingual (English/Chinese) dataset of riddles involving personification and metaphors.
  • RiddleSense (RS): Requires complex commonsense reasoning and understanding of figurative language.
Figure 5: Detailed workflow of the first two steps of Metacognitive Prompting applied to a brain teaser example.

Key Results and Insights

The experimental results across five base LLMs (Llama3, Qwen series, GPT-3.5, and GPT-4) demonstrate the efficacy of the MP approach:

  • Performance Gains: MP consistently outperformed Standard Prompting, Automatic Prompt Engineer (APE), and CoT. For GPT-3.5-turbo, MP improved performance over CoT by 5.00% on Sentence Puzzles and 10.07% on Word Puzzles.
  • Surpassing Human Benchmarks: GPT-4 with MP achieved scores of 98.33% and 98.96% on BRAINTEASER tasks, notably higher than the human average of ~91-92%.
  • The Critical Role of Reflection: The "Reflecting" step allows models to resolve inconsistencies between reasoning steps and final answers, effectively mitigating logical hallucinations.
  • Scalability: The benefits of MP scale with model size; the performance gap between MP and standard methods widens as the base model becomes more capable.

Conclusion and Future Work

Metacognitive Prompting represents a significant step toward endowing LLMs with the creative flexibility required for lateral thinking. While highly effective, the method still faces challenges such as "misdirection" (where models are overwhelmed by distractors) and "overthinking" (where reflection leads a model to reject a correct initial intuition). Future research will focus on integrating broader cultural and historical contexts to further enhance the model's ability to solve riddles dependent on external knowledge.

Summary updated

BibTeX
@inproceedings{bai2025mp,
  author = {Tian Bai and Yongwang Cao and Yan Ge and Haitao Yu},
  title = {{MP: Endowing Large Language Models with Lateral Thinking}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2025},
  doi = {10.1609/aaai.v39i22.34514},
}
Figure