Language models that forecast stock volatility from earnings-call transcripts appear to use qualitative information, such as management tone, guidance and analyst pushback, that balance sheets leave out. A recent audit found that much of this is an illusion. Transcripts from the same company look nearly identical to a text encoder, and models mostly learn to recognize which company is speaking and recall its usual volatility. A training-free rule that predicts each firm's historical average does as well as the best text model, or better. We propose a specific kind of synthetic data that breaks this shortcut: splices. A splice is a transcript assembled from real segments so that the company's identity comes from one call and the quarter-specific content comes from another. Because every segment is real, each splice inherits a real outcome label for its content. A model that only recognizes the company gives the same prediction for every splice from that firm. A model that reads the content ranks the splices the way their real outcomes rank.
The note makes two claims. The first is that splices work as a validation instrument: they expose models that rely on company identity, which ordinary held-out validation cannot do, because identity-driven models score well there. The second is that splices, used with care, work as training augmentation that pushes models toward content. Because the synthetic data is the thing being relied on, we also evaluate the splices themselves. We check whether a discriminator can tell them apart from real calls, and whether they leak identity. That check determines whether any gain is real or an artifact of the synthesis.
What prior work establishes and leaves open
The motivating result is Same Company, Same Signal. On its Dense Earnings Call (DEC) dataset, which covers 90 US firms over 20 consecutive quarters and includes the market-timing field that older benchmarks omitted, the zero-parameter same-ticker mean (STPEV(Mean)) reaches a mean squared error (MSE) of 0.242 on post-call volatility. The best transcript-embedding model reaches 0.249 and zero-shot GPT-4o reaches 0.349. Model predictions correlate at about 0.85 with the ticker mean, and a random vector per ticker matches full-text models. That paper diagnoses the problem but does not propose a way to force or certify content use. That second step is the one we take. It also shows that older benchmarks such as MAEC and EC have corrupted labels for over 60% of samples, which constrains us to use DEC as the primary testbed. It also means we cannot compare directly with DialogueGAT, whose MSE of 0.4530 on extended MAEC was measured on labels of that kind.
Two lines of work already put synthetic text into financial forecasting, and they are the closest neighbours. MTCA builds counterfactual earnings calls by swapping sentences in from financial news or older calls. It keeps swaps that a gradient-influence score (TracIn+) favours and reports a 14.2% average accuracy gain. Its swaps are not designed to separate identity from content, it did not control for the ticker shortcut, and it never evaluates the synthetic samples themselves. Overcoming the Modality Gap in Context-Aided Forecasting has a large language model (LLM) write explanatory text conditioned on future values. It keeps only text that improves a frontier model's forecast, and its model reaches a 71% win rate over the numbers-only Chronos model on the verified test split. The authors acknowledge verifier bias: the kept text reflects what the verifier already believes. Our splices avoid both problems. They require no generator, because all content is real, and no verifier, because labels come from real outcomes. The price is a modelling assumption, discussed below.
Two further papers supply our methods. Synthetic Data as Validation shows that procedurally built synthetic cases used for model selection rather than training improve real performance. On liver-tumour segmentation, the Dice overlap score rose from 26.7% to 34.5% in-domain. We transfer that role to finance. Benchmarking the Fidelity and Utility of Synthetic Relational Data supplies discriminative detection: train a classifier to separate real from synthetic, and treat accuracy above 50% as evidence of detectable flaws. We apply that test to splices. Forecasting Earnings Surprises from Conference Call Transcripts finds that signal is spread across the call and tracks the density of forward-looking statements. This tells us where the splice boundary should fall: identity-bearing material (prepared boilerplate, names, product vocabulary) goes on one side, and forward-looking and question-and-answer content goes on the other.
Core Claim
We assume post-call volatility for firm $f$ in quarter $q$ decomposes approximately as $y_{f,q} = \mu_f + r_{f,q}$. Here $\mu_f$ is the firm's baseline, estimated from training quarters as in STPEV, and $r_{f,q}$ is the quarter-specific residual that intangible, qualitative content could explain. A splice $s(f, q \to q')$ keeps firm $f$'s identity-bearing segments from quarter $q$ and inserts the content segments from quarter $q'$. Its label is $\mu_f + r_{f,q'}$.
Cross-firm splices $s(f \to g, q')$ place firm $g$'s content inside firm $f$'s identity segments. They test directly whether predictions follow the identity or the content. The claim is threefold. First, a content-sensitivity score computed on held-out splices ranks candidate models by how well they predict real residuals on unseen quarters, better than real-validation MSE does. Second, selecting models by this score yields a model that beats STPEV on real DEC test quarters. Third, training on splices adds value beyond simply training on residual targets.
Research Program
We first build the splicer. Each DEC transcript is segmented into prepared remarks and question-and-answer. Forward-looking sentences are tagged with guidance-token rules in the style of Koval et al. We then separate identity-bearing spans (named entities, firm-specific product and segment names, recurring boilerplate identified by high cross-quarter overlap within a firm) from content spans. Three splice types come out of this: within-firm swaps across quarters, cross-firm transplants, and a control type that swaps identity spans only.
Before any modelling, we audit the splices. A gradient-boosted discriminator in the style of discriminative detection is trained on real versus spliced calls. If its accuracy stays well above 50%, we smooth the boundaries (sentence-level seams, discourse-marker repair) and re-audit. An identity probe then asks whether a ticker classifier trained on real calls recovers the identity-source firm or the content-source firm from cross-firm splices. That tells us which spans actually carry identity. Only splices that pass both audits are used.
The central test of the additive assumption is empirical. For within-firm splices it is close to tautological. For cross-firm transplants it is not, because firm $g$'s content may mean something different inside firm $f$. We therefore report results separately by splice type, and we restrict cross-firm transplants to firms in the same sector as a sensitivity check.
We then define the content-sensitivity score. For each held-out firm, it is the Spearman correlation between a model's predictions on that firm's splice set and the borrowed labels $r_{f,q'}$, averaged over firms. An identity-only model scores zero by construction. The score is used for model selection, and separately as a training signal: splices are added as regression examples, together with a pairwise ranking loss within each firm.
Baselines
The naive reference is STPEV(Mean) from Same Company, Same Signal, recomputed on our splits using training quarters only. Its reported value is MSE 0.242 on DEC. Beating it means a lower real-test MSE, with a firm-level bootstrap interval that excludes zero improvement. Because DEC has only 1,800 calls, we report that interval rather than point estimates. A numbers-only extension adds the previous quarter's realized volatility and pre-call volatility to the ticker mean. This is the strongest fair version of the argument that the numbers are all you need, and it has no reported result on Lacuna.
The strongest existing text model is the best transcript-embedding regressor from the same paper, with a reported MSE of 0.249. We rerun it with the same encoder, splits and hyperparameter search budget as our models. The most dangerous competitor is that same regressor trained on the residual target $y - \mu_f$ rather than raw volatility. This removes the identity shortcut from the objective without any synthetic data, and every splice arm must beat it. We also run MTCA-style sentence-swap augmentation with TracIn+ filtering MTCA at a matched number of synthetic examples. No reported result was found on Lacuna for MTCA on DEC. Zero-shot GPT-4o, at MSE 0.349, is included as the frontier-LLM reference.
Two ablations isolate the key ingredient. The first uses LLM paraphrases of real calls with unchanged labels, giving synthetic volume without decoupling identity from content. The second uses splices with shuffled borrowed labels, giving decoupling without correct content labels. If either ablation matches the full method, the gain comes from the extra data rather than from separating identity and content.
Why not just…?
Why not just train on residual targets and skip synthetic data? This is the real threat, and it may win. Subtracting the firm mean removes the easiest shortcut. However, residual training on 1,800 calls still lets the encoder pick up quarter-specific style that correlates with a firm's time trend. It also offers no way to check content use at selection time. The residual-target baseline is trained and selected at the same tuning budget. If it matches splice training on real-test MSE, we report that splices add nothing as training data and keep only the validation-instrument claim. That claim is tested independently by asking whether splice-based selection picks better models than residual-MSE selection.
Why not just anonymize company names? The 0.937 within-firm embedding similarity reported by Ding Yu et al. suggests identity lives in recurring vocabulary and structure, not just names. The identity-only control splices and the ticker probe measure how much identity survives masking. If masked text already defeats the probe, anonymization is enough, and we will say so.
Why not just use MTCA or generate-then-verify augmentation? MTCA's swaps change content but leave identity intact, so a ticker-reading model can still ignore them. Generate-then-verify inherits the verifier's beliefs, as Overcoming the Modality Gap itself notes. The matched-budget MTCA arm and the paraphrase ablation test whether our decoupling beats generic synthetic text.
Why not just ask a stronger LLM? GPT-4o zero-shot already trails the ticker mean (0.349 against 0.242). Nothing in the file suggests that scale alone fixes identity reliance, though we include it as a reference.
Proposed Experiments
There are two splits on DEC: the last four quarters of every firm held out, and 20 firms held out entirely. The held-out-firm split has no firm mean available, so STPEV is replaced by a sector mean, and it tests whether content use transfers to new firms. The metrics are MSE, the fraction of residual variance explained on real test calls, and the content-sensitivity score on held-out splices.
The first decisive check is whether splice-selected models beat both STPEV and the residual-target baseline on real test MSE, with bootstrap intervals. The second is whether, across about 50 candidate models, the content-sensitivity score correlates with real residual accuracy more strongly than validation MSE does. The third is whether cross-firm transplant predictions follow the content source rather than the identity source. Throughout, the discriminator audit must stay near 50% accuracy, so that no result can be explained by detectable seams.
As a secondary transfer test, we apply the same splicing to the MSCI transcripts task, where Hierarchical FinBERT reaches 76.56% accuracy in Forecasting Earnings Surprises. We ask whether splice selection helps there too, with the caveat that its identity shortcut has not been audited.
A clean negative result would also be informative. If no model scores content sensitivity above zero while matching STPEV, earnings-call text carries little residual signal for volatility at this scale. That would be a sharper conclusion than any published so far.
References
- Same Company, Same Signal: The Role of Identity in Earnings Call Transcripts
- DialogueGAT: A Graph Attention Network for Financial Risk Prediction by Modeling the Dialogues in Earnings Conference Calls
- Multi-Domain Transformer-Based Counterfactual Augmentation for Earnings Call Analysis
- Overcoming the Modality Gap in Context-Aided Forecasting
- Synthetic Data as Validation
- Benchmarking the Fidelity and Utility of Synthetic Relational Data
- Forecasting Earnings Surprises from Conference Call Transcripts