Many databases hold long streams of measurements, such as industrial sensors, patient vitals, or financial ticks. Users want to ask them questions in plain language, like "when did pressure rise by more than 20 units within five minutes and then stay flat?" Scanning every raw point is expensive at scale. So practical systems first query a compressed summary of the stream, and only look at raw data for a few candidate windows. The problem is that the summary is a sparse temporal sample: it discards detail. Whatever it discards can make the system miss real events or accept false ones, and current systems cannot tell when this has happened. The best current system, Sonar-TS, turns the stream into fixed symbolic strings, has a language model write SQL (the standard language for querying tables) over those strings, and then re-checks the candidates in Python. Its own ablation shows the average score collapsing from 0.6144 to 0.27 without that check. The summary alone is not trustworthy, and a candidate missed by the summary is never recovered.
We propose to replace the fixed symbolic summary with a multi-resolution summary that stores exact bounds for each time window (minimum, maximum, first and last value, and timestamps of the extremes) at several window sizes. We also change what the language model writes. Instead of free SQL, it writes a query in a small library of temporal predicates such as "exceeds", "rises within", and "precedes". A compiler turns each predicate into a pair of SQL queries. One finds windows where the predicate is certainly true given the bounds. The other finds windows where it might be true. Only the windows in between, where the summary cannot decide, are refined to finer resolution and eventually to raw points. The query itself decides where the data gets sampled densely. Within the predicate library, no true event is ever missed, and the number of raw points read grows with the number of ambiguous windows rather than with the length of the history.
Core Claim
Treating a stored temporal summary as a lossy sample, and reasoning explicitly about what it can and cannot decide, fixes two failures together. Mean-based symbolic summaries miss events. Downstream-only verification is costly. For a predicate $P$ and a window $w$ with stored summary $s(w)$, the compiler produces a lower evaluation $L_P(s(w))$ and an upper evaluation $U_P(s(w))$, each 0 or 1, such that
$$L_P(s(w)) \le P(x_w) \le U_P(s(w)),$$
where $x_w$ is the raw signal in $w$. Windows with $L_P = 1$ are accepted and windows with $U_P = 0$ are rejected, both without touching raw data. Windows with $L_P = 0$ and $U_P = 1$ are split into their child windows at the next finer level, and the test is repeated. This is branch-and-bound pruning, which here means discarding whole windows whose bounds already settle the answer. It applies to temporal predicates that have monotone bounds on windowed extremes.
Take "the signal rose by more than $\Delta$ within five minutes" as an example. Stored extremes give a sound necessary condition for a window: max minus min must reach $\Delta$, with the time-ordered extremes consistent with a rise. They also give a sufficient condition: an ordered pair of extremes that are close enough in time. Composed patterns such as "surge then plateau" become conjunctions of these predicates over adjacent intervals.
The claim is empirical as well as structural. At a matched budget of raw points read, this approach should beat Sonar-TS's search-then-verify pipeline on event recall. It should keep that recall as the history grows by orders of magnitude, while the number of raw points read stays roughly flat.
Relation to Prior Work
Sonar-TS is the closest work and provides the problem setting, the benchmark, and the main negative result that motivates us. We keep its two-stage idea and change three things. Its summaries are SAX strings (symbolic aggregate approximation: window means mapped to letters). They fix one granularity offline and average away the extremes that threshold and slope questions depend on. Our summaries keep exact extremes at many granularities. Its verification is a separate Python filter over every candidate. Our refinement happens inside the query plan and only touches undecided windows. Finally, its search stage can drop a true event without anyone noticing. Our search stage cannot, within the predicate library.
The agentic spatio-temporal system of From Queries to Insights wraps a language model in a plan-act-observe loop with schema inspection and self-correction. On LBSN check-in data it raises accuracy from 28.6% to 91.4%. It motivates us to keep an agent loop for turning phrases like "night shift" or "shortly after" into concrete intervals. Its data, though, is discrete point events, and it has no notion of resolution.
Video frame selection supplies the refinement idea. A.I.R. zooms in on promising intervals with repeated vision-language model checks. Adaptive Keyframe Sampling trades off relevance to the question against coverage of the whole video. SlowFast-LLaVA documents that uniform sampling ignores the question and wastes budget on static stretches. We borrow the coarse-to-fine zooming. The difference is that our zoom decision comes from exact bounds rather than learned similarity scores. That gives a no-miss guarantee, and it avoids the latency of repeated model calls that A.I.R. incurs.
Schema-linking work such as KaSLA and AutoLink assumes the tables faithfully describe the world. We relax that assumption: our summary tables are marked as lossy samples at specific resolutions. EHRSQL constrains the design in two ways. Temporal questions dominate real clinical use (93.2% of its queries). And abstaining under uncertainty matters. Our three-valued evaluation gives a principled abstention signal whenever raw data is unavailable for an undecided window.
Research Program
The work has three parts.
The first part is the summary index. For each series we store a table per resolution level, with window sizes growing by a fixed factor. Each row holds the window's start and end, its min and max, their timestamps, its first and last values, and its count. This is ordinary relational data, so any SQL engine can host it. Storage is a geometric series over levels and totals only a constant multiple of the finest summary level.
The second part is the predicate library and its compiler. We define roughly a dozen primitives:
- threshold exceedance and staying above a value,
- bounded rise and fall within a duration,
- flatness within tolerance,
- precedence with a time gap,
- counting of crossings.
For each primitive we write the lower and upper SQL predicates over one summary row and over pairs of adjacent rows, and we prove soundness by case analysis. Morphological patterns that fall outside the library, such as oscillation, are handed to Sonar-TS-style Python verification on the undecided windows. They are reported separately, with no guarantee claimed.
The third part is the language interface. The language model is prompted with the predicate library and with metadata describing the summary tables, including their resolutions. It writes a predicate program rather than raw SQL. An agent loop in the style of From Queries to Insights repairs execution errors and grounds vague durations, and the cold-start prompting from Sonar-TS is reused. The refinement loop is deterministic and makes no language-model calls, which is where most of the expected latency savings come from.
Baselines
The naive reference point is plain Text-to-SQL over the stored data. For this we use MAC-SQL as reported by Sonar-TS, which scores 0.1611 average on full NLQTSBench and 0.4767 on the short setting. We rerun it with our language-model backbone so the comparison is not confounded by model choice.
The strongest existing method is Sonar-TS itself. Its reported average score on NLQTSBench is 0.6144, with online latency of around 22 seconds, according to Sonar-TS. It is the only system in the file built for natural-language search over long series, and it already beats ChatTS (0.3016) and MAC-SQL there. We run it with the same backbone model, the same cold-start prompts, the same per-query budget of raw points read, and the same number of language-model calls. We report its published number next to our rerun. Beating it means a higher average score at equal or lower raw-point budget. It also means strictly higher event recall on the long-horizon extensions described below, where we expect missed events to be Sonar-TS's dominant error.
Two ablations isolate the key ingredient. The first swaps our extreme-preserving summaries for mean-only summaries at identical window sizes, keeping everything else fixed. If the gains survive, the extrema were not what mattered. The second keeps our summaries but verifies every candidate window, Sonar-TS style, instead of refining only the undecided ones. This separates the recall benefit of bounds from their cost benefit. We also report Sonar-TS without verification (0.27 reported) as a floor for coarse-only search.
Why not just…?
Why not just use finer or multi-scale SAX in Sonar-TS? This is the most serious competitor. A finer symbolic index might recover most missed events without new machinery. But SAX letters encode window means, so a brief spike inside a window is averaged away at any fixed granularity coarser than the spike. More resolution only shrinks the problem at a proportional storage cost. We test this directly by giving Sonar-TS a multi-level SAX index with the same storage footprint as our pyramid and comparing recall on injected transient events of varying width. If matched multi-scale SAX equals our recall, the contribution reduces to the refinement policy, and we will say so.
Why not just let an agent write SQL over the raw table? At about 12,000 points per series, a database can scan raw data cheaply, and an agentic loop like From Queries to Insights may do well. This is a real threat on the existing benchmark, and we will report it honestly. The argument for summaries is about scale. We therefore extend NLQTSBench histories to $10^6$ and $10^8$ points by concatenating real background segments, and measure raw points read and latency alongside accuracy. We expect raw scanning to stay accurate but grow linearly in cost, while our method should stay nearly flat.
Why not just put the series into a long-context language model, or sample it uniformly more densely? The file already argues against both. ChatTS reaches only 0.3016 on NLQTSBench and 0.2491 on the Lite version, per Sonar-TS. Evidence from video, such as Flexible Frame Selection beating a 32-frame uniform baseline with 13.5x fewer FLOPs, shows that denser uniform sampling pays for redundancy while still risking misses of short events. We include uniform downsampling at matched raw-point budgets as a sanity check.
Proposed Experiments
The main evaluation is on NLQTSBench and NLQTSBench-Lite using the benchmark's average score, together with three further measures: event-window recall, raw points read per query, and wall-clock latency. The decisive check is the long-horizon extension. We inject events of controlled width, from shorter than the finest summary window up to many windows long, and plot recall against history length for our method, matched multi-scale SAX Sonar-TS, and raw-scan agents. The proposal succeeds if our recall is flat and matches raw scanning while reading orders of magnitude fewer points. It fails if Sonar-TS's recall at matched storage is statistically indistinguishable from ours.
A second check measures how often the summary cannot decide: the fraction of undecided windows at each level for each primitive. This tells us whether refinement stays confined to narrow bands in practice or spreads across whole series on noisy data like SMD.
A third, smaller study reuses EHRSQL-style temporal templates over MIMIC-III vitals rebuilt as summary pyramids. It tests whether three-valued evaluation gives a better abstention signal than the entropy threshold EHRSQL uses, measured by the F1 score for correctly answering versus correctly abstaining when raw data for undecided windows is withheld.
References
- Sonar-TS: Search-Then-Verify Natural Language Querying for Time Series Databases
- From Queries to Insights: Agentic LLM Pipelines for Spatio-Temporal Text-to-SQL
- A.I.R.: Enabling Adaptive, Iterative, and Reasoning-based Frame Selection For Video Question Answering
- Adaptive Keyframe Sampling for Long Video Understanding
- SlowFast-LLaVA: A strong training-free baseline for video large language models
- Knapsack Optimization-Based Schema Linking for LLM-Based Text-to-SQL Generation
- AutoLink: Autonomous Schema Exploration and Expansion for Scalable Schema Linking in Text-to-SQL at Scale
- EHRSQL: A Practical Text-to-SQL Benchmark for Electronic Health Records
- Flexible Frame Selection for Efficient Video Reasoning