Asking Before Assuming: Solver-Rewarded Clarification for Underspecified Contracts

When a language model reads a contract that does not quite settle a question, it rarely says "that depends on a fact I don't have." It usually fills the gap with a plausible background assumption and answers confidently. Work on formal legal reasoning has measured this. Know Your Limits translates non-disclosure agreements into logic checked by the Z3 solver, a program that decides whether a conclusion strictly follows from given premises. It finds that models often reach the benchmark-correct label by injecting unstated assumptions. In up to 52.5% of Qwen 2.5-72B's "logical" conclusions, the answer does not follow from the logic the model wrote. The solver itself conservatively answers "not determined" on the same cases. We argue this is not only a legal-reasoning failure. It is the legal face of single-turn optimization bias: models tuned to give one helpful answer per turn learn to skip clarifying questions. Grounding Gaps in Language Model Generations shows that off-the-shelf models are 77.5% less likely than humans to ask or acknowledge, and that preference tuning makes this worse (correlation R = −0.79 between DPO steps and human-AI grounding agreement for Mistral-7B).

The proposal is to train a contract-reading agent over whole conversations so that it asks the user for exactly the missing facts that would settle the question, and then answers only what follows. The reward comes from a solver, not from an LLM judge. The key device borrows from coalition game theory. Each candidate unstated assumption is treated as a "player." A set of assumptions "wins" if adding it to the formalized contract makes the solver's verdict definite. The smallest winning sets tell us which questions are worth asking. Capping their size at a small constant $k$ keeps finding them polynomial, which is the standard bounded-coalition trick. This produces a turn-by-turn credit signal that is cheap, hard to game, and tied to logical necessity rather than to what sounds helpful.

Core Claim

Legal hallucination on underspecified contracts can be reduced by training an agent across whole conversations to ask about decisive assumptions. This approach should beat three alternatives: answering directly (which launders assumptions), refusing (which is honest but useless), and asking generically (which is prompted or judge-rewarded clarification). The new ingredient is the reward. It combines a solver check at the end of the dialogue with per-question credit computed from bounded-size minimal winning coalitions of assumptions.

Formally, let $A = \{a_1,\dots,a_n\}$ be the candidate unstated assumptions for a contract $C$ and hypothesis $h$. Define a simple game with value

$$v(S) = \mathbb{1}\big[\,C \wedge S \vdash h \;\;\text{or}\;\; C \wedge S \vdash \neg h\,\big], \quad S \subseteq A,$$

where $\vdash$ means the solver proves the statement. A minimal winning coalition is a set $S$ with $v(S)=1$ whose proper subsets all have value 0. An assumption is pivotal if it belongs to some minimal winning coalition of size at most $k$. Enumerating all such coalitions takes $O(n^k)$ solver calls, following the bounded-size argument of Task Allocation Via Coalition Formation Among Autonomous Agents and Coalition Formation with Bounded Coalition Size. The dialogue reward is

$$R = \mathbb{1}[\text{final answer correct and solver-entailed by } C \cup E] \;-\; \lambda\, q \;-\; \mu\, q_{\text{non-pivotal}},$$

where $E$ is the set of facts elicited from the user, $q$ is the number of questions asked, and $q_{\text{non-pivotal}}$ is the number of questions about assumptions that belong to no small minimal winning coalition.

We should be honest about what the game-theoretic framing buys. Minimal winning coalitions of this game are essentially the "minimal correction subsets" that Know Your Limits names as the needed fix but never operationalizes. The coalition view does not add a new solution concept. It supplies the size-capped tractability argument, and it turns "which question matters" into a checkable property that can serve as a training signal.

Research Program

The first component is data. We start from the 610-example ContractNLI evaluation set re-annotated in Know Your Limits, which separates strict formal entailment from pragmatic judgments and includes minimal pairs. We select cases where the strict label is "not determined" but the pragmatic label is definite. Legal annotators then write down the background assumptions that license the pragmatic label, and those become the hidden knowledge of a simulated user. The minimal pairs let us flip hidden facts so that the correct answer changes. An agent that ignores the user and guesses the usual default will therefore fail about half the time on the flipped side. This construction is the main annotation cost. Given the expense of legal labeling documented by CUAD, we budget for a few hundred cases rather than thousands.

The second component is the reward oracle. Each contract is autoformalized once, offline, into Z3. Candidate assumptions come from two sources: the annotators' assumptions and model-proposed distractors. We enumerate minimal winning coalitions with $k \le 3$. Autoformalization is noisy; the same paper reports a 25.5% Z3 program-synthesis error even for its best model. To bound the effect of this noise, we hand-verify formalizations on a held-out subset and compare training with verified versus automatic formalizations.

The third component is training. We follow the hierarchical design of ArCHer. A critic scores whole utterances ("ask about X", "answer Entailed") by long-term value, and a token-level policy writes the words. We chose it because it reaches peak Twenty Questions performance in under 1,000 samples versus over 100,000 for PPO, and solver calls and simulated dialogues are not free. MTPO, which learns from preferences over whole dialogues, is run as an alternative optimizer. Its preferences are generated by the same solver-grounded reward. Base models are open 8B-class instruction models, so that a small model trained this way can be compared against frontier models used as prompted clarifiers.

Baselines

The naive reference point is the base model answering in one turn from the contract. It is scored on the flipped minimal pairs, where guessing the default fails. We report its accuracy and its laundering rate, meaning the fraction of answers not entailed by the contract plus elicited facts. The conservative reference point is the pure symbolic pipeline from Know Your Limits, which answers "not determined" whenever the solver cannot decide. It never launders but is useless on these cases. Beating it means higher accuracy at a laundering rate no worse than a few points above zero.

The strongest prompted competitor is a frontier model instructed to ask clarifying questions before answering, with the same simulated user and the same question budget. For reference, Claude 3.5 Sonnet reaches 83.0% accuracy in LLM-based formal reasoning on this ContractNLI subset (Know Your Limits), but that figure is for single-turn answering. We found no reported result on Lacuna for interactive assumption elicitation on ContractNLI or any legal benchmark.

The strongest trained competitor is a CollabLLM-style agent. CollabLLM trains a collaborator with rewards from forward-simulated future turns scored by an LLM judge, and reports an 18.5% average task-performance gain on general collaboration tasks. We reimplement it on the same base model, data, simulator, and number of rollouts, replacing only our solver reward with its judge-scored reward. This comparison isolates whether solver grounding matters.

The single-turn factuality competitor is HIPO, iterative DPO with a likelihood term that trains verbatim statute recall. It reports a Non-Hallucinated Statute Rate of 38.35% for GLM4-9B versus 16.0% for GPT-4o on LegalHalBench. We adapt it to our data as one-shot preference tuning toward solver-correct answers. This tests the prediction that single-turn factuality tuning lowers laundering only by increasing refusals.

Two ablations target the key ingredient. The first trains with the terminal solver reward alone, setting $\mu = 0$. The second replaces pivotal-set credit with credit for random assumptions, matched in count. If the per-question credit from minimal winning coalitions does not beat the first ablation in question efficiency or sample efficiency, the coalition component is decorative and we will say so.

All systems receive identical contracts, simulator, maximum turns, and tuning budget measured in rollouts. Beating the field means strictly higher accuracy on flipped pairs, a laundering rate at or below the symbolic baseline plus a small tolerance, and fewer questions than the prompted clarifier.

Why not just…?

Why not just prompt a strong model to ask clarifying questions? The grounding-gap study found that prompted grounding produced human-AI agreement with Cohen's kappa consistently below 0.2, with questions asked at the wrong moments (Grounding Gaps). CollabLLM likewise reports that prompted proactivity is overeager. Still, a frontier model may well ask about the right clause in a narrow NDA domain. We treat this as a real threat and settle it through the question-precision metric: the fraction of questions that hit pivotal assumptions, at matched budgets.

Why not just use an LLM judge as the multi-turn reward, as CollabLLM does? This is the most serious alternative. The file documents reward hacking against judges in ArCHer and MTPO, and a judge can reward fluent, plausible assumptions, which is exactly the laundering we target. The head-to-head comparison in Baselines decides it. If judge-rewarded training matches ours on laundering rate, the solver adds little, and the contribution shrinks to the dataset.

Why not just let the solver say "not determined" and refuse? This is honest but unhelpful. MASLegalBench shows the same pattern, with high refusal rates from models such as DeepSeek-v3.1 on incomplete context. Our design only wins if it converts refusals into correct answers, and the symbolic baseline measures exactly that.

Why not just list every assumption the model would make and ask the user to confirm them all at once? This avoids the multi-turn problem entirely, but it grows linearly with distractor assumptions and pushes the relevance judgment onto the user. We include it as a one-turn baseline and report user burden, measured as the number of assumptions shown. Coalition-based credit should matter most when there are many candidate assumptions and few are pivotal.

Proposed Experiments

The decisive experiment runs on the flipped minimal pairs. A model that has learned to ask should keep its accuracy when the hidden fact contradicts the usual default, whereas direct answerers and HIPO-style tuning should drop sharply on that side. We also track laundering rate, questions per dialogue, and question precision.

A second check tests whether coalition credit, rather than the terminal reward alone, drives efficiency. We plot accuracy against rollouts for the full reward and the $\mu = 0$ ablation, and vary $k$ from 1 to 3 to show how much tractability costs in missed pivotal sets.

A third check measures robustness to formalization noise by comparing training on verified versus automatic Z3 programs. A fourth tests transfer: we run the trained agent, without solver access, on MASLegalBench with simulated users holding withheld case facts, and measure whether refusals turn into correct answers. That final test indicates whether the agent learned a habit of asking about decisive facts, or only a solver-specific trick.