Polysemy as a Hidden Cause of the Synthetic-to-Real Gap in Time-Series–Text Retrieval

Models that link sensor readings to text are increasingly trained on text written by a large language model (LLM), because real paired data is scarce. One example is a retriever that, given a weather-station record, finds the matching incident report, or the reverse. LLM-written descriptions are verbose and tidy. They spell out "mean arterial pressure" and "maximum daily temperature." Real operators and clinicians write "MAP," "temp," "P," and "charges." Those are short tokens whose meaning depends on the domain, the channel, and the field. We argue that much of the drop such retrievers suffer when moved from synthetic to real text comes from this one cause. Synthetic text removes ambiguity from the vocabulary, so the model never learns to resolve it.

Our claim is that the continuous signal itself is the cheapest disambiguating context available. A heart-rate trace and a precipitation trace both labelled "rate" look nothing alike. The text alone cannot say which sense is meant, but the signal often can. We propose to generate synthetic training text that deliberately contains real-world shorthand collisions. For each such text we pair "sense distractors": texts that use the same token in a different sense, attached to different signals. We then train the time-series–text aligner to reject those distractors. The test is whether this closes the synthetic-to-real retrieval gap on genuinely human-written text, and specifically on queries that the text alone cannot resolve.

Core Claim

The proposal builds most directly on TRACE. TRACE pre-trains a time-series encoder with per-channel identity tokens and aligns it to text notes using contrastive learning at both the whole-series and per-channel level. It then uses the aligned embeddings for retrieval-augmented generation (RAG), meaning it retrieves stored items and feeds them to a downstream model. Concretely, it prepends retrieved historical patterns as continuous "soft tokens" to a frozen forecaster. TRACE reports "channel confusion" as a core failure: text fails to map to the right sensor channel. It trains on high-quality real pairs and mines hard negatives without regard to word sense.

The second source is KnowPAT. KnowPAT shows that dense text retrievers pull in passages where a polysemous keyword ("MAC") is used in the wrong sense. It trains an LLM to discount such "distractor knowledge" through a preference objective, but it operates purely on text.

Our claim combines the two and adds the synthetic-to-real axis. When the aligner is trained only on LLM-generated text, injecting sense-collision distractors as contrastive negatives teaches the encoder to use signal shape to resolve ambiguous tokens. Generic text noise does not do this, and neither does text-only acronym expansion. Neither of the two prior works tests this. TRACE never trains on synthetic text or measures polysemy. KnowPAT never conditions disambiguation on a non-text modality.

Research Program

The work has three parts.

The first part is a measurement study. We train a TRACE-style aligner on synthetic text only and test it against real human-written text. Real queries are labelled by whether they contain domain-overloaded tokens, and, if so, whether the intended sense is recoverable from the surrounding text or only from the signal. This tells us how much of the gap concentrates in ambiguous queries. If it does not concentrate there, the core hypothesis is wrong and we stop.

Two prior results motivate this study. Detecting the Clinical Features of Difficult-to-Treat Depression trained an extractor on clean GPT-3.5 clinical notes and saw it collapse on real notes, scoring 0.30 F1. Injecting telegraphic shorthand recovered 0.60 F1. Separately, Overcoming the Modality Gap in Context-Aided Forecasting concedes that its LLM-generated contexts in CAF-7M are more formal and complete than real text. Both suggest shorthand matters, but neither isolates sense ambiguity from generic messiness.

The second part is the method, which we call collision-aware synthetic pairing. For each training series we prompt an LLM to write a description using the abbreviations and shorthand that actually appear in the target domain's channel names and reports. Examples include PhysioNet's "HR," "MAP," and "NIMAP," and GHCN's "PRCP" and "TMAX." We then build a small collision lexicon from those names across domains: tokens that appear with different referents in different channels or datasets. For every synthetic text containing a lexicon token, we sample distractor texts that use the same token in another sense and attach each to the series of that other sense. Training uses TRACE's contrastive loss with these distractors up-weighted:

$$\mathcal{L} = -\log \frac{e^{s(x,t^+)/\tau}}{e^{s(x,t^+)/\tau} + \sum_{t \in N_b} e^{s(x,t)/\tau} + \lambda \sum_{t \in N_c} e^{s(x,t)/\tau}}$$

Here $x$ is a time-series window, $t^+$ its paired text, and $s$ the cosine similarity between their embeddings. $\tau$ is a temperature, $N_b$ the ordinary in-batch negatives, $N_c$ the sense-collision distractors, and $\lambda$ their weight. The loss is applied at both the whole-series and the channel level, so channel confusion caused by a shared token is penalized directly.

Real text never enters training. That keeps the synthetic-to-real question clean. It also respects the warning in Rethinking Multimodal Time-Series Forecasting Evaluation: rankings obtained on synthetic benchmarks flip on leakage-free real data, so all of our evaluation is on real text.

The third part is the downstream use. The improved retriever feeds TRACE's soft-token RAG pipeline for forecasting and classification. This checks whether better sense resolution at retrieval time carries through to the task.

We hold the time-series encoder fixed throughout. Rethinking LLMs for Irregular Time Series Classification in Critical Care found that on real ICU data the encoder design mattered far more than the alignment strategy: a 12.8% AUPRC gain from the encoder against 2.9% from alignment. That result constrains what we may claim. Our contribution is about which text gets aligned to which signal. It is not a claim that LLM alignment beats specialized models.

Baselines

The naive reference is a TRACE-architecture aligner trained on plain LLM-written descriptions with in-batch negatives only. This is the configuration whose synthetic-to-real gap we measure, and every method below shares its encoder, text tower, synthetic series, and training steps.

The upper reference is TRACE trained on real paired data, as published. On the NOAA event-report and GHCN station benchmark, TRACE reports approximately 90% top-1 label matching accuracy. On Time-MMD, its RAG setup reports a 4.56% gain in classification accuracy and a 4.55% reduction in forecasting error (TRACE). We treat the gap between the naive synthetic-trained model and this real-trained model as the quantity to close.

The strongest synthetic-only competitor is generic telegraphic noise injection, following the Difficult-to-Treat Depression paper: dropped articles and random abbreviation and shorthand substitution, with no collision structure. A second competitor is knowledge-infused generation in the style of CLINGEN. CLINGEN conditions LLM generation on knowledge-graph entities and on writing styles sampled from real data, and it improved PubMedBERT by 8.7% (base) and 7.7% (large) in few-shot clinical tasks. We adapt it by conditioning descriptions on domain ontology entries and on style exemplars. Both competitors receive the same number of generated texts and the same hyperparameter search budget as our method.

The key ablation keeps the shorthand-rich synthetic text but removes the collision negatives ($\lambda = 0$, with distractors also dropped from $N_b$). This isolates whether the gain comes from contrasting senses or merely from exposure to shorthand.

A further baseline adds a KnowPAT-style text-only reranker on top of the naive retriever. We cannot reuse KnowPAT's reported CPQA numbers here, because its tasks are text QA. The comparison is run on our benchmarks.

Beating these baselines means two things. Our method must exceed noise injection and the ablation on real-text top-1 accuracy and Recall@10 on the signal-resolvable ambiguous subset, across at least three training seeds. It must also not lose on unambiguous queries.

Why not just…?

Why not just inject telegraphic noise, as the depression-notes paper did? This is the most serious threat, because noise injection already adds abbreviations. Its abbreviations are sampled without regard to collisions, though. The model therefore sees ambiguous tokens but never a training signal saying that the same token paired with a different signal shape is wrong. The ablation above is designed to settle this. If noise-only matches the full method on the signal-resolvable subset, the collision mechanism is unnecessary and we would report it that way.

Why not just expand acronyms with an LLM or a glossary before retrieval? When the surrounding text determines the sense, this probably works as well as anything, and we expect it to tie our method on the text-resolvable subset. Our claim is restricted to queries that text alone leaves undetermined, such as a channel header "P" or a two-word operator note. In those cases an expander has to guess the sense, while the signal-conditioned retriever can use the trace's shape. We will run an expander using a strong instruction-tuned LLM with the domain glossary in context. Reporting results separately by subset shows exactly where each approach wins. If the expander wins on both subsets, the proposal reduces to a recommendation for preprocessing.

Why not skip alignment and ensemble a time-series foundation model with a text LLM? On TimesX, a plain average ensemble of TimesFM-2.5 and Gemini-2.0-Flash reached the best MASE of 0.619. That beat complex agentic code-revision methods at 0.713 (Rethinking Multimodal Time-Series Forecasting Evaluation). An ensemble cannot retrieve signals from text or text from signals, which is the task here. For the downstream forecasting check we include this ensemble anyway, so that any claim about RAG benefits is measured against it.

Why not just collect a little real paired text? Where real pairs exist, they should be used, and the real-trained TRACE serves as our ceiling for that reason. The proposal targets settings where they do not exist, which is what drives the use of CAF-7M-scale synthetic corpora. We additionally report how many real pairs a naive model needs to match our synthetic-only model, which converts our gain into a data-equivalent figure.

Proposed Experiments

The primary setup trains on GHCN station series with LLM-generated descriptions and tests retrieval against real NOAA incident reports, which are human-written. Metrics are top-1 label matching and Recall@k. Results are stratified into three subsets: unambiguous queries, ambiguous queries resolvable from text, and ambiguous queries resolvable only from the signal. Stratification uses a lexicon match followed by two-annotator labelling of whether the text alone fixes the sense.

The second setup is cross-domain collisions on Time-MMD. Its nine domains are pooled into a single retrieval index, so that overloaded tokens like "rate," "index," and "load" collide across agriculture, health, energy, and economy. Training uses synthetic text, and testing uses Time-MMD's real filtered reports. Downstream we measure soft-token RAG forecasting MSE and classification accuracy against TRACE's reported 4.56% and 4.55% gains.

The third setup is a clinical channel-name stress test on PhysioNet 2012. Its 41 abbreviated variables are queried with clinician-style shorthand, and we measure channel-level retrieval accuracy under its 85.7% missingness. We use the irregularity-aware mTAND-style encoder that the critical-care study found essential. No reported result was found on Lacuna for text-to-channel retrieval on PhysioNet 2012, so the naive model sets the reference there.

The decisive check is a pre-registered pattern across these setups. Our method should beat noise injection and the no-collision ablation on the signal-resolvable subset. It should tie the acronym expander on the text-resolvable subset. And it should recover a substantial fraction of the gap to real-trained TRACE on NOAA. If the measurement study shows the synthetic-to-real gap is not concentrated in ambiguous queries, or if the ablation matches the full method, the hypothesis is refuted. Both outcomes are informative for anyone building cross-modal retrievers from synthetic text.