Introduction
While Large Language Models (LLMs) excel at text generation, they often falter when tasked with deep causal reasoning. A common failure mode is the generation of spurious causal associations, where the model relies on superficial patterns rather than rigorous logic. For instance, given a cause involving favorable sailing conditions, an LLM might generically conclude a fleet "made good time," ignoring specific historical or environmental factors that lead to a different effect.
To address this, we introduce LEGO (Multi-agent Collaborative Framework), which treats LLMs as modular components assigned to specialized roles. This collaborative approach mimics human expert reasoning, moving beyond unidirectional "cause-to-effect" jumps to a more holistic, bidirectional analysis.
The LEGO Framework
The LEGO architecture is divided into two primary modules designed to bridge the gap between raw input and logical explanation.
1. Fine-grained World Knowledge Integration (FWKI)
This module focuses on gathering the necessary facts to link a cause to an effect. It utilizes three specialized agents:
- Cause and Effect Analysts: These agents perform bidirectional reasoning. Instead of just looking forward from the cause, the Effect Analyst works backward from the result to identify necessary preconditions.
- Knowledge Master: Acting as a targeted knowledge retrieval system, this agent synthesizes fine-grained facts to answer specific queries from the Analysts, effectively serving as a more efficient alternative to traditional external search methods.
2. Iterative Feedback and Refinement (IFR)
Once knowledge is gathered, the explanation is polished through a multi-turn dialogue:
- Explainer: Synthesizes the Analysts' findings into a coherent narrative.
- Critic: Evaluates the draft for "Observation Feedback" (ensuring it doesn't just restate the input) and "Commonsense Feedback" (verifying logical consistency).
The agents iterate until the explanation meets the required standards of rigor and conciseness.
Experimental Evaluation
The framework was tested on two major benchmarks: WIKIWHY, which requires multi-hop reasoning grounded in Wikipedia, and e-CARE, which focuses on conceptual causal facts.
Key Results
- Performance Gains: On the WIKIWHY dataset, LEGO achieved an F1 score of 0.666, significantly outperforming standard GPT-3.5 (0.615) and search-augmented baselines like ReAct (0.631).
- Human Preference: In blind evaluations, human judges found LEGO’s explanations satisfactory 75.3% of the time, a notable improvement over the 71.3% baseline.
- Ablation Studies: Removing either the Knowledge Master or the Critic led to a sharp decline in performance, confirming that the multi-agent synergy is essential for overcoming reasoning shortcuts.
Discussion and Insights
A critical finding of this research is that LLMs often treat causal explanation as a "concatenation" task—simply gluing the cause and effect together with a transition word. LEGO breaks this habit by forcing the model to investigate the underlying mechanics. Furthermore, using an LLM as an internal "Knowledge Master" proved more effective than external search, as the model could better synthesize relevant context than a raw search engine.
While LEGO offers superior accuracy, it does introduce higher computational costs and latency due to the multi-agent interactions. Future work will focus on optimizing these interactions to reduce noise and improve efficiency in non-linear causal scenarios.
Summary updated