Quantifying Cross-Query Contradictions in Multi-Query LLM Reasoning
Lists
Only you see this
Abstract
Large language models frequently produce mutually inconsistent answers when reasoning over multiple related queries. We study case-file logical consistency: maintaining a globally satisfiable belief state across interdependent queries. We introduce a benchmark of 390 multi-query reasoning instances with entailment/contradiction/unknown labels and propose set-level metrics including Case Satisfiability Rate, Contradiction Density and Revision Cost. Our solver-augmented approach extracts commitments, verifies global satisfiability and performs counterexample-guided repair. Across four reasoning domains, our method substantially reduces cross-query contradictions (SetCons: 0.56 to 0.94) while preserving per-query accuracy, demonstrating that global coherence is critical for robust multi-query reasoning.
BibTeX
@inproceedings{salla2026quantifying,
author = {Rohit Kumar Salla and Ramya Manasa Amancherla and Manoj Saravanan},
title = {{Quantifying Cross-Query Contradictions in Multi-Query LLM Reasoning}},
booktitle = {LLM Reasoning @ ICLR},
year = {2026},
url = {https://openreview.net/forum?id=v9jBDyc72l},
}Loading related content...