Abstract
Large Language Models with chain-of-thought prompting, such as OpenAI-o1, have shown impressive capabilities in natural language inference tasks. However, Multi-hop Question Answering (MHQA) remains challenging for many existing models due to issues like hallucination, error propagation, and limited context length. To address these challenges and enhance LLMs’ performance on MHQA, we propose the Self-Guiding prompting Finite State Machine (SG-FSM), designed to strengthen multi-hop reasoning abilities. Unlike traditional chain-of-thought methods, SG-FSM tackles MHQA by iteratively breaking down complex questions into sub-questions, correcting itself to improve accuracy. It processes one sub-question at a time, dynamically deciding the next step based on the current context and results, functioning much like an automaton. Experiments across various benchmarks demonstrate the effectiveness of our approach, outperforming strong baselines on challenging datasets such as Musique. SG-FSM reduces hallucination, enabling recovery of the correct final answer despite intermediate errors. It also improves adherence to specified output formats, simplifying evaluation significantly.
Introduction
Multi-Hop Question Answering (MHQA) requires a model to find and connect multiple pieces of information across different documents to reach a final answer. While Large Language Models (LLMs) are powerful, they struggle with MHQA due to three main issues: error propagation, where a mistake in an early reasoning step cascades; "Lost-in-the-Middle" reasoning, where the model loses track of the original question (Answer Drift); and format mismatch, where models fail to follow structured output instructions.
To address these challenges, we introduce the Self-Guiding Finite State Machine (SG-FSM). This zero-shot prompting framework treats MHQA as a formal state-transition process, ensuring the LLM focuses on one manageable task at a time to prevent cognitive overload and hallucinations.
Methods
The SG-FSM framework follows a formal state-transition logic. Instead of a linear prompt, the system functions as an automaton that guides the model through specific functional roles.
The process is divided into two main stages:
- SG-FSM1 (Iterative Discovery): The model loops through four roles: a Decomposer (breaks down the question), a Searcher (finds sub-answers), a Revisor (fixes formatting or logical errors), and a Terminator (decides if the task is complete).
- SG-FSM2 (Final Verification): A Summarizer reviews the entire reasoning path, checks for consistency, and performs a final self-correction to produce the definitive answer.
Experimental Setup
The framework was evaluated on three high-quality MHQA benchmarks:
- HotpotQA: Standard 2-hop questions with 10 candidate paragraphs.
- 2WikiMultiHopQA (2WikiQA): 2-hop questions with complex relationship types.
- Musique: The most difficult benchmark, containing 2-4 hop questions and 20 candidate paragraphs, designed to prevent "shortcut" reasoning.
Key Results and Insights
SG-FSM significantly outperformed strong baselines, including Direct prompting, Chain-of-Thought (CoT), and Self-Prompted CoT (SP-CoT), across both GPT-3.5-turbo and Qwen-72B models.
Performance on Challenging Benchmarks
On the Musique dataset, SG-FSM2 achieved an F1 score of 40.5 (GPT-3.5) and 48.5 (Qwen), nearly doubling the performance of baseline methods in some cases. Despite being a zero-shot method, it outperformed many few-shot approaches by enforcing better logical structure.
Reliability and Format Robustness
By forcing the model to return to a "Terminator" state after every hop, the FSM acts as a logical anchor, preventing Rationale Drift. Furthermore, the dedicated "Revisor" state allowed the system to achieve 100% format accuracy across multiple tests, whereas standard CoT often produced unusable outputs. The framework also improved "Joint" scores (correct answer + correct evidence), indicating more faithful reasoning compared to models that might give the right answer for the wrong reasons.
Limitations and Future Work
While effective, SG-FSM introduces inference overhead because it requires multiple turns of dialogue for state transitions. It also relies on the model's ability to follow instructions for each state, which may be challenging for smaller models (e.g., 7B parameters). Future research could explore integrating this state-based logic directly into the training or fine-tuning phase of LLMs to embed structured reasoning into the model's weights.
Summary updated