Data-Grounded Interpretation Trees for Ambiguous Visualization Requests

When someone asks a data tool to "show the gross trend of movies by year," there is usually no single right chart. "Gross" might be domestic or worldwide revenue, "trend" might be a line or a bar chart, and the years might be raw or binned. Current language-model systems either pick one reading silently or invent several readings from what they have seen in training. The second habit causes a quieter failure: the model proposes interpretations that the table cannot support, such as a column that does not exist, an average over a categorical label, or a filter that returns nothing. This proposal treats an ambiguous request as a search over partially filled chart specifications. A chart specification here is the small structured record that interactive dashboards already save and share, listing which columns, filters, transforms, mark type and axis mappings are in use. The candidate values at each step come from the actual table, not from the model's imagination. Each partial specification is executed against the data before it is expanded further.

The two-sentence version is this. Visualization systems already represent a chart as data plus a serialized state record, so an ambiguous request can be decomposed into a tree whose nodes are partial state records. Each node can be executed and checked against the real table, which grounds the enumeration of interpretations in the data rather than in model priors. We expect this to recover more of the valid readings of an ambiguous request, especially rare ones, while emitting fewer impossible ones than sampling many complete charts from a model and filtering afterward.

Core Claim

Interactive visualization systems describe a view as a deterministic function of a dataset and a separate configuration state. This is how PyGWalker turns drag-and-drop interactions into portable configuration records, how HiGlass shares linked genomic views as structured records, and how ManyEyes put views into bookmarkable URLs. In that line of work the state record is only a storage format. It records what is set and has no model of whether the setting matches what the user meant.

We claim the same record is the right unit for decomposing an ambiguous request. Its fields have a natural dependency order: columns come first, then mark types compatible with those column types, then channel mappings, then transforms. A partial record can be executed to check type compatibility, non-empty filter results, and whether an aggregation makes sense. Following that order and checking each prefix against the data gives a decomposition tree in which every branch is a data-supported interpretation.

The claim is narrow and falsifiable. On requests with several simultaneous ambiguities, data-grounded enumeration over partial state records should give higher recall of valid interpretations and a lower rate of impossible specifications than two alternatives at the same model-call budget. The first alternative is a linear decomposition model. The second is sample-then-filter over complete charts. The advantage should grow with the number of ambiguous fields.

Research Program

The system builds on three pieces of prior work, each playing a distinct role. nvBench 2.0 supplies the task and the evidence for the gap. It injects ambiguity into seed charts at a data layer (column choice, filters) and a visualization layer (mark, channel mapping), and it trains Step-Text2Vis, which reasons through five fixed stages in sequence. That sequence commits to one path, so it cannot keep two column readings alive while it reasons about chart type. Our tree keeps the same stage order as the dependency order of the state record, but branches at every stage.

VERDICT supplies the grounding principle. It showed that in text question answering, brainstorming interpretations before looking at the corpus produces "pseudo-interpretations" the corpus does not contain, and that retrieving first and verifying against passages raises grounded precision. Our proposal moves that reversal from passages to tables. Candidate values for a field are proposed by matching the request against the table schema and column statistics, and the language model then ranks them. The model does not generate them freely. VERDICT also clusters its interpretations into a flat set. Our interpretations carry dependencies between fields, so a flat set would discard information that matters for pruning.

Tree of Clarifications and ProbTree supply the tree machinery. Tree of Clarifications recursively splits ambiguous questions and prunes branches by self-verification. ProbTree traverses a query tree and gates each node by confidence. Both verify with the model's own judgment or with retrieval. We replace self-verification with execution, following Analytica, which grounds leaf claims by running code in a sandbox.

Concretely, the system has three parts. A proposer is either the released Step-Text2Vis model or a prompted backbone. At a node holding a partial state record, it ranks schema-derived candidates for the next field. An executor renders the partial record against the table and rejects candidates on hard checks: a missing column, a quantitative encoding on a nominal field, an empty filter result, or an aggregate over an identifier. A scorer keeps a per-branch plausibility from the proposer's rankings. We deliberately do not merge branches into a single blended specification. Analytica shows that linear probability synthesis avoids tipping points for scalar truth values, but a chart is discrete: it is either a bar or a line. We therefore keep ranked sets of complete specifications and use scores only to order and truncate them. This sidesteps the tension between continuous confidence and discrete chart syntax rather than resolving it, and we state that plainly.

The output is one serialized object, the interpretation tree itself, in which every leaf is a valid shareable state record. A collaborator who opens the link can move between sibling interpretations instead of receiving one silently chosen chart. We treat this sharing property as a design consequence and check only that every leaf round-trips through serialization and rendering. We do not claim it as an empirical contribution.

Baselines

The naive reference point is a prompted GPT-4o that receives the table schema and the request and returns one specification, or k specifications listed in one call. nvBench 2.0 reports that Step-Text2Vis beats this by over 20% on both F1@3 and F1@5. No absolute GPT-4o number appears in the file, so we will rerun it on the same splits.

The strongest existing method is Step-Text2Vis. It reports F1@3 of 81.50% and F1@5 of 80.88% on nvBench 2.0 (nvBench 2.0). We will run the released model under the benchmark's own protocol to reproduce these figures before comparing. Our main configuration will use Step-Text2Vis as the proposer, so that any gain is attributable to the search and grounding rather than to a different model. Beating it means a higher F1@5 on the full test set and a clearly larger gain on the highest ambiguity levels (K = 4 and K = 5), with model calls capped at the same number per query for every method. Beam search over its stages will be given the same cap.

The baseline that most threatens the claim is sample-then-filter. It draws N complete specifications from the same proposer at temperature, deduplicates them, and discards those that fail the same executor checks. N is chosen to match our tree's average number of model calls.

Two ablations isolate the ingredients. The first keeps the tree but lets the model propose branch values freely, with no schema-derived candidates and no execution checks. This is essentially Tree of Clarifications adapted to chart records, and it isolates grounding. The second keeps schema-derived candidates and execution checks but drops the tree. It forms the Cartesian product of per-field candidates, filters it, and ranks the result with the proposer. This is a VERDICT-style flat enumeration, and it isolates the hierarchy.

Why not just…?

Why not just sample many complete charts and filter the broken ones? This is the most serious alternative, because the executor checks can be applied after the fact as easily as during search. Our reason to expect it to fall short is coverage, not validity. LLM-based Query Expansion Fails for Unfamiliar and Ambiguous Queries shows that models expanding ambiguous queries collapse toward the most popular interpretation and lose recall on long-tail readings. Repeated sampling of whole charts should repeat the dominant column and mark choice and rarely reach the product of several rare choices. The comparison is direct: recall of valid interpretations stratified by K at matched calls. If sample-then-filter matches the tree at K = 4 and K = 5, the coverage argument is wrong, and the contribution reduces to the executor checks.

Why not just enumerate data-grounded candidates per field and filter their product, with no tree? This is also a real threat. For small schemas the product is cheap, and a good ranker may recover everything the tree finds. The tree should help in two cases. One is when conditional dependencies matter for ranking, for example when the plausible marks depend on which column was chosen. The other is when the product grows too large to rank within budget. The flat ablation settles this question. If it matches the tree on F1 and on cost, we will report data-grounded enumeration as the finding and treat the hierarchy as an implementation convenience.

Why not just ask the user a clarifying question? CLAIM picks clarifying questions by information gain over disagreement among several models. The pragmatic-dialogue work on DS-1000 plotting tasks raises Pass@1 from 68.38% to 79.44% with a cooperative persona. Clarification is complementary to our method rather than a substitute. It still requires knowing which interpretations exist, and asking about every ambiguous field creates fatigue. The tree offers a natural target for a single question: the field whose split carries the most plausibility mass. We test this in the secondary experiment below.

Proposed Experiments

The primary experiment runs every method and ablation on the nvBench 2.0 test set. We report F1@3 and F1@5 overall and per ambiguity level. We also report an invalid-specification rate, the fraction of returned charts that fail execution checks or reference fields absent from the table, measured before any filtering so that baselines are not credited with checks they did not perform. Model calls and wall-clock time per query are reported alongside.

The decisive check targets hallucination. We construct a perturbed copy of the nvBench 2.0 tables in which columns the model's priors expect are renamed to rarer but valid synonyms, or removed while a near-synonym is kept, for example deleting gross and keeping box_office_revenue. Gold interpretations are adjusted to match. Free generation, and sample-then-filter before its filter, should fabricate the expected column names or collapse onto the surviving obvious column. Schema-derived candidates cannot produce a missing column at all. The measurable claims are three. Our method's recall should drop much less than the baselines' recall under perturbation. The free-proposal tree ablation should show a fabrication rate close to the untreated baselines. The gap should widen with K. If none of these hold, the grounding argument fails.

A secondary experiment uses the tree as a clarification planner on nvBench 2.0 with a simulated user who holds one gold interpretation. We ask one question about the highest-mass split and compare top-1 accuracy against asking a question chosen by a CLAIM-style disagreement score and against not asking. We note the known limitation that simulated users stand in for people here, as they do in CLAIM. A small transfer check maps the 155 annotated DS-1000 plotting tasks onto declarative records where possible. It tests whether grounded enumeration narrows the gap that dialogue closes there, starting from the 68.38% Pass@1 baseline.