Abstract
Dialogue Commonsense Multi-choice Question Answering (DC-MCQ) is a complex task requiring models to navigate "clue labyrinths" and handle "option saturation." Traditional linear reasoning often fails to capture the nuances of multi-choice scenarios. We introduce ReX-GoT (Reverse Exclusion Graph-of-Thought), a framework that leverages the human cognitive strategy of elimination. By first identifying and excluding unreasonable options, the model refines its contextual understanding. Subsequent graph-based reasoning validates the remaining choices. ReX-GoT demonstrates superior performance, achieving a 39.44% F1 score improvement over standard GPT-3.5 benchmarks.
Introduction: The Challenges of Dialogue Reasoning
In Natural Language Processing, DC-MCQ tasks are particularly difficult because they lack a fixed number of correct answers—a phenomenon known as option saturation. Models cannot simply identify the single "best" fit; they must evaluate every option independently and in relation to the dialogue history.
Furthermore, clues are often buried deep within the conversation or the options themselves, creating a clue labyrinth. Standard Large Language Models (LLMs) using linear Chain-of-Thought (CoT) prompting frequently overlook subtle interrelations or are misled by plausible-sounding distractors.
The ReX-GoT Framework
The ReX-GoT framework addresses these hurdles through a three-step structured reasoning process.
1. Reverse Option Exclusion
Instead of searching for correct answers immediately, the model first identifies unreasonable options and provides justifications for their exclusion. This "reverse" step generates "negative clues," which help the model narrow its focus and clear the noise from the dialogue context.
2. Error Analysis and Branching
The model then performs a deeper analysis of the remaining candidates. It constructs a reasoning structure where the problem serves as a "central node" and potential answers act as "branch nodes." Each option is compared against the dialogue "main stem" to verify its logical consistency.
3. Graph-of-Thought Synthesis
Finally, the model employs a Graph-of-Thought (GoT) approach to synthesize these reasoning paths. Unlike linear CoT, GoT allows for the simultaneous exploration of multiple logical branches. A voting mechanism evaluates the consistency across the graph to select the final set of correct answers.
Experimental Evaluation
The researchers tested ReX-GoT on several benchmarks, including CICERO, CICEROv2 (where all questions are multi-choice), and CICERO-Multi.
The results were decisive:
- Zero-Shot Superiority: On CICEROv2, ReX-GoT (using Flan-T5-11B) outperformed the previous state-of-the-art by 17.67% in F1 score.
- GPT-3.5 Performance: When integrated with GPT-3.5, the framework yielded a massive 39.44% F1 score increase compared to standard prompting methods.
- Ablation Studies: Removing either the exclusion step or the graph structure led to significant performance degradation, confirming that both components are vital for handling complex multi-choice reasoning.
Conclusion
ReX-GoT demonstrates that "negative knowledge"—knowing why something is wrong—is a powerful catalyst for identifying what is right. By moving from linear reasoning to a non-linear Graph-of-Thought, models can better navigate the complexities of human dialogue. While the multi-step process introduces some computational overhead, the significant gains in accuracy and reasoning depth suggest that structured, eliminative logic is a promising direction for future LLM development in complex domains.
Summary updated