Systems that describe tables in text increasingly compute the facts first, by running a small program over the table, and only then ask a language model to put those facts into words. The writing step is usually treated as the easy part. This note argues that it often is not. Turning "A's score is greater than B's" into a sentence that a paragraph about B needs ("B trailed A") requires the writer to reverse argument order and switch from "greater" to its opposite word in the same breath. Language models are known to fail on exactly this kind of small, easy-looking inversion even when they handle harder problems. We propose to measure how often verbalization reverses the direction of an already-verified fact when the required wording departs from the program's canonical argument order. We also propose a fix: a cheap deterministic step that resolves direction and roles before the model writes. It decides which entity is the grammatical subject, which comparative word to use, and whether the sentence is negated, and hands the model a plan where none of that remains to be inferred.
The opportunity matters because the most faithful table-to-text pipelines in the file already put correct logic in front of the writer and still lose fidelity at the last step. The Table-to-Logic-to-Text system of Automatic Logical Forms improve fidelity in Table-to-Text generation reaches 75% human-judged fidelity, and its authors report that the GPT-2 writer confuses "greater" with "smaller" and "maximum" with "minimum" even when the logical form is correct. If direction errors are a structured, predictable failure of realization rather than random noise, they can be removed by construction instead of filtered after the fact.
Core Claim
We make three claims. First, realization errors on verified facts are concentrated in a few direction-bearing choices: argument order under a change of focus or voice, the polarity of comparative and superlative words, and negation. Second, these errors grow with how far the requested surface form sits from the logical form's canonical order, and they persist in modern instruction-tuned open models, not only in small fine-tuned realizers. Third, resolving those choices in a deterministic plan before generation, rather than asking the model to infer them, removes most of these errors at no measurable cost in fluency. The new element is not syntactic control as such, which Using Structured Content Plans for Fine-grained Syntactic Control in Pretrained Language Model Generation already shows works on AMR graphs. It is the identification of the direction-bearing subset of realization decisions as the place where faithfulness is lost, together with computing those decisions from the executed program.
How This Sits Against Prior Work
Three lines of work motivate the proposal by locating the gap. The Table-to-Logic-to-Text pipeline uses an executable logical form plus false-candidate rejection, then an unconstrained GPT-2 writer, and it documents comparative reversals in that writer. Enhancing Content Planning for Table-to-Text Generation with Data Understanding and Verification improves content selection on ROTOWIRE from 41.00 to 44.39 F1 without matching gains in surface metrics, which suggests that realizing a good plan is a separate bottleneck. Beyond Traditional Benchmarks: Analyzing Behaviors of Open LLMs on Data-to-Text Generation finds semantic errors in 76% to 86% of open-LLM outputs hidden behind fluent prose. None of these isolates realization from reasoning by holding the computed fact fixed and varying only the required wording. That isolation is our design.
Two papers supply the method. Lee et al. show that BART follows node-level voice, tense and pronoun tags with high fidelity: tense F1 is 0.979 against 0.691 untagged, and the flip rate in the forced-unnatural "flipped" setting is 0.498 against 0.048. Tags also slightly raise BLEU, which argues against a fluency tax. Their tags concern grammatical form, though, not the truth-relevant direction of a relation, and their inputs are single AMR graphs with no execution. Think Fast, Talk Smart: Partitioning Deterministic and Neural Computation for Structured Health Text Generation supplies the partitioning principle. Deterministic code handles comparisons and rankings, and a bounded writer only verbalizes. Swapping the comparison layer for an LLM raises numeric error from 0.7% to 16.9%, and swapping the writer interface raises numeric error to 4.8% and attribution error to 10.7%. That last number is the closest existing evidence that the writer step alone leaks errors. But that paper's writer interface is a hand-built health schema, and it never asks which linguistic decisions inside the writer cause the leak. We take its partition one level down, into the sentence.
One paper constrains the design. Can Large Language Models Always Solve Easy Problems if They Can Solve Harder Ones? shows that models fail on strictly easier sub-problems through variable mismatch and distraction, with 7B models at 50% to 70% consistency. This predicts that an "easy" inversion in wording will not be reliably solved by a model that can solve the harder reasoning. It also warns us that adding many tags could become a distraction in itself, so the plan must stay minimal.
Research Program
The work has three parts.
The first part is a diagnostic suite, which we call focus-shifted Logic2Text. Logic2Text: High-Fidelity Natural Language Generation from Logical Forms provides 10,753 gold logical forms over Wikipedia tables across seven operation types. Each one can be executed, so the truth of any rewording can be checked. For every comparative, superlative, ordinal and unique form, we generate realization requests that keep the fact fixed but demand a different surface shape. The variants are: subject set to the non-canonical argument, passive voice, negated framing ("did not exceed"), and nominalization. A short multi-sentence variant asks for a paragraph centered on one entity that must state several facts involving it. That variant is where focus shifts arise naturally rather than by instruction. We define "canonical distance" as the number of inversions (argument swap, polarity swap, negation) separating the requested form from the logical form's native order. Faithfulness is then measured as a function of that distance.
The second part is the direction-locked plan. A deterministic resolver reads the executed logical form and the requested focus or voice. It outputs a compact plan with three slots: which entity is the subject, the exact comparative or superlative word with polarity already applied ("fewer than", "lowest"), and whether a negation is present. It also outputs each argument's literal value. The writer receives this plan alongside the table excerpt. The resolver is a few hundred lines of rules over the seven Logic2Text operation types. It is not domain-specific software in the sense of the health-domain contracts, because it is keyed to logical operators rather than to a particular dataset. We implement the writer in two forms. One is a fine-tuned T5-large, the model family PLOG found more reliable on logical syntax than BART. The other is prompted open instruction models of the kind Kasner and Dusek studied.
The third part tests transfer to settings where the facts are not hand-given. We run the same writer behind an automatic logical-form parser, reusing the Table-to-Logic-to-Text parser and rejection step on Logic2Text. We also run it on SciGen, where comparative and argmax/argmin statements dominate and SciGen: a Dataset for Reasoning-Aware Text Generation from Scientific Tables reports only about 39% correct statements from BART and T5. For SciGen we compute candidate comparisons with code over the table, following the partition principle, and ask whether locking direction raises correctness on top of correct computation.
Baselines
The naive reference point is end-to-end table-to-text: the same writer models given the table and highlighted cells with no logical form. This tells us how much of the gain comes from computing facts at all. Logic2Text reports correctness near 20% without the logical form and over 82% with it, so we expect a large gap here. That gap is not our contribution, which is why it is only the floor.
The strongest existing method is the Table-to-Logic-to-Text pipeline, which reports 75% human fidelity on Logic2Text-style data against 44.9% for end-to-end baselines. When gold logical forms are given, the matching reference is the fine-tuned GPT-2 of Logic2Text at 82.4% human-judged correctness. We rerun both pipelines with the same logical forms our system receives, gold in one setting and parsed in the other. We replace only the writer, keep the same model size, give both arms the same fine-tuning data and steps, and tune both on the same development split. Beating it means two things. On the standard, unshifted Logic2Text test set, fidelity must be at least as high, with no loss in BLEU or human fluency. On the focus-shifted suite, the drop in faithfulness as canonical distance grows must be significantly smaller, and we pre-register a target of halving the drop from distance zero to distance two.
The second strong competitor is an open instruction model that receives the executed fact in plain words plus the focus request, but no direction-locked plan. This is the bounded writer of the health-text paper, transplanted to Logic2Text. It is the baseline most likely to be close, so it gets the same models, prompts that differ only by the plan block, and an equal budget of prompt-tuning trials.
We include two ablations that isolate the key ingredient. One supplies Lee et al.-style grammatical tags (voice, tense, pronoun) without the direction slots. It tests whether any syntactic plan helps or whether direction specifically matters. The other supplies the direction slots but lets the model choose the comparative word itself, which separates argument-order locking from polarity locking.
Why not just…?
Why not just give a strong modern LLM the computed fact and trust it to word it correctly? This is the real threat. The reported comparative reversals come from a fine-tuned GPT-2, and a larger instruction model may rarely make them at canonical distance zero. The evidence that the problem persists is indirect: replacing the health system's writer interface raised attribution error to 10.7%, and the hard-to-easy results show that strong models still swap variables on simpler sub-problems. Our focus-shifted suite settles the question directly. If open instruction models show a flat faithfulness curve across canonical distance, the method is unnecessary for them, and we will report it as a result about small fine-tuned realizers only.
Why not just generate several outputs and reject unfaithful ones with a checker? Cycle-consistency reconstruction and the BLEC keyword metric from Logic-Consistency Text Generation from Semantic Parses can flag direction errors; BLEC correlates +0.66 with human logic judgments. Rejection sampling is therefore a fair competitor. It costs several generations per fact, though, and its recall on passive and negated paraphrases is unknown, since keyword matching is weakest exactly where surface order departs from the logical form. We run BLEC-filtered best-of-k at k equal to 4 and 8 against our single-sample writer. We report faithfulness at matched compute and test whether the two combine.
Why not just use templates for each operator? Templates are faithful by construction. The question is what they cost in fluency and coherence in multi-sentence text. We include a template realizer and compare human fluency and coherence ratings on the entity-centered paragraph variant. If templates match our system on human preference, the case for a neural writer collapses, and we will say so.
Why not just ask for chain-of-thought before writing? Thinking aloud about direction is the model doing the inversion itself, and Evaluating Intermediate Reasoning of Code-Assisted Large Language Models for Mathematics finds intermediate steps often unsound even when outputs look right. We include a chain-of-thought writer arm at matched tokens.
Proposed Experiments and Decisive Checks
Faithfulness on all Logic2Text conditions is judged by executing the gold logical form against a human reading of the sentence. We use three-way human annotation on 600 sampled outputs per system, with BLEC and an LLM judge as automatic proxies. Kasner and Dusek report r = 0.93 between GPT-4 error counts and human error counts, which justifies the judge, but we calibrate it on our own human sample before relying on it. Fluency is measured by BLEU and human ratings. On ToTTo we report BLEU and PARENT only as a regression check against the reported BERT-to-BERT scores of 44.0 BLEU and 52.6 PARENT from ToTTo: A Controlled Table-To-Text Generation Dataset.. On SciGen we use human correctness against the reported ~39% best case. We found no reported result on Lacuna for any system on a focus-shifted or voice-controlled variant of Logic2Text, so that suite has no external number to beat; its baselines are run by us.
Three outcomes would be decisive. If the unplanned writers' faithfulness does not fall with canonical distance, the core claim is false. If the grammatical-tags-only ablation recovers most of the gain, then the contribution is ordinary syntactic control and not direction locking. If our gains vanish behind the automatic parser on Logic2Text and SciGen, then parsing errors dominate and realization is not the bottleneck at current parser quality. We would then redirect the work toward joint content selection, the gap left open by the reliance on pre-selected cells in Table-to-Logic-to-Text and PLOG.
References
- Automatic Logical Forms improve fidelity in Table-to-Text generation
- Using Structured Content Plans for Fine-grained Syntactic Control in Pretrained Language Model Generation
- Enhancing Content Planning for Table-to-Text Generation with Data Understanding and Verification
- Beyond Traditional Benchmarks: Analyzing Behaviors of Open LLMs on Data-to-Text Generation
- Think Fast, Talk Smart: Partitioning Deterministic and Neural Computation for Structured Health Text Generation
- Can Large Language Models Always Solve Easy Problems if They Can Solve Harder Ones?
- Logic2Text: High-Fidelity Natural Language Generation from Logical Forms
- SciGen: a Dataset for Reasoning-Aware Text Generation from Scientific Tables
- Logic-Consistency Text Generation from Semantic Parses
- Evaluating Intermediate Reasoning of Code-Assisted Large Language Models for Mathematics
- ToTTo: A Controlled Table-To-Text Generation Dataset