Beyond Wasserstein Proximity: Finding Better Transfer Sources

Gordei Verbii, Georgii Killinkarov, Vladimir YafarovarXiv 2026 · arXiv

TL;DR

Counter-intuitively, pretraining a model on data that is dissimilar to your target can lead to significantly better forecasting performance than using the most similar data. In the volatile world of cryptocurrency, the authors show that selecting the "farthest" sources via Wasserstein distance can improve forecasting skill by over 20 percentage points for Transformers, challenging the standard "proximity-first" heuristic in transfer learning.

What Problem Does This Paper Address?

When we don't have enough data to train a model from scratch—a common issue for newly listed financial assets—we often turn to transfer learning: pretraining a model on a "source" asset before fine-tuning it on our "target."

The million-dollar question is: Which source should we pick? The conventional wisdom is to choose the most similar data. Researchers often use metrics like the Wasserstein distance to measure the "proximity" between two data distributions, assuming that the closer the source is to the target, the better the transfer.

This paper challenges that assumption. The authors argue that marginal similarity (how much the data "looks" the same) doesn't necessarily capture predictive utility. Sometimes, a source that is geometrically distant might actually be more informative because it contains a broader range of variation that helps the model learn the underlying relationships more robustly.

Methods

The researchers focused on forecasting five-day cryptocurrency volatility. Their approach involved several key components:

  1. Measuring Distance: They used the Gaussian-Wasserstein ($W_2$) distance to quantify the difference between the feature distributions of different crypto assets over 30-day windows.
  2. Source Selection Policies: They compared three main strategies for picking a pretraining source:
  3. Nearest $W$: Pick the asset with the smallest $W_2$ distance to the target.
  4. Farthest $W$: Pick the asset with the largest $W_2$ distance.
  5. Forward Pilot: A "brute force" validation method where they test all potential sources on a small "pilot" window of target data and pick the best performer.
  6. Model Architectures: They tested six different model families, ranging from simple linear models (DLinear) and recurrent networks (LSTMs, GRUs) to complex MLPs and Transformers.
  7. Evaluation: Performance was measured using "persistence-relative skill"—essentially, how much better the model performs compared to a simple baseline that assumes tomorrow's volatility will be the same as today's.

Datasets

The study used a "frozen panel" of daily spot USDT bars from Binance for ten major cryptocurrency assets:

  • Established: BTC, ETH, BNB, XRP, SOL.
  • Newer/Diverse: SEI, JTO, ONDO, SUI, TIA.

The data spanned from August 2022 to September 2026 (per the paper's specific historical panel boundaries), providing a mix of long-established histories and short, uneven tracks for newer listings.

Key Insights and Ideas

  • The "Broader Variation" Advantage: The authors provide a theoretical regression example showing that if a source has a wider range of informative variation (i.e., it's "farther" away), it can estimate the shared relationship more accurately than a "near" source with cramped data.
  • Learner-Specific Sensitivity: The benefit of choosing distant sources isn't universal—it depends heavily on the model. While Transformers and MLPs saw massive gains from "farthest" sources, simpler recurrent models (like GRUs) were less sensitive to the geometric distance of the source.
  • Selection vs. Quality: A crucial "aha" moment is that better selection does not equal better forecasting. While Transformers benefited the most from smart source selection, they were still outperformed in absolute terms by compact recurrent models (LSTMs and GRUs).
Figure 1: This chart shows that for MLPs and Transformers, picking the "Farthest" source (orange) significantly outperforms picking the "Nearest" source (blue) relative to random transfer.

Key Results

  • Farthest Beats Nearest: For the Transformer model, selecting the three farthest sources improved skill by 20.90 percentage points over the nearest sources. For the MLP, the gain was 12.52 percentage points.
  • Pilot Selection Works: Across all six model families, using a "forward pilot" to validate sources before the final test led to better mean skill than picking a source at random.
  • Recurrent Models Rule: Despite the selection gains for larger models, the GRU-32 and LSTM-16 models achieved the lowest absolute Mean Squared Error (MSE), even outperforming the standard HAR (Heterogeneous Autoregressive) econometric baseline.
Figure 3: The left panel shows that while Transformers and MLPs gain a lot from selection, their absolute error (MSE) remains higher than the simpler LSTM and GRU models.

Limitations and Future Work

The authors are careful to note that these findings are exploratory and based on a fixed historical panel. Key limitations include:

  • Fixed Budgets: The models were trained for a fixed number of epochs; different adaptation lengths might change the results.
  • Computational Cost: While "geometric screening" (using $W_2$ distance to narrow down candidates) shows promise, the study didn't measure actual runtime savings.
  • Generalization: The results are specific to the 10 assets and the specific time period studied.

Future work could involve testing these selection rules across different market regimes, varying the adaptation length, and exploring "multi-source" learning where models are pretrained on a diverse mix of sources rather than just one.

Figure 4: Retrospective screening shows that for MLPs and Transformers, using Wasserstein distance to filter candidates before pilot testing can provide an incremental performance boost.

Summary updated

Abstract

Source selection often favors similar data, yet marginal proximity need not identify the most useful pretraining source. We motivate a conditional alternative through domain-adaptation theory and a regression example in which broader source variation improves estimation despite increasing Wasserstein distance. We examine five-day cryptocurrency volatility using ten Binance spot assets, six forecaster families, three seeds, and shared temporal cutoffs. Selecting the three farthest rather than nearest Gaussian-Wasserstein sources improves mean persistence-relative skill by 12.52 percentage points for MLP and 20.90 for Transformer, with positive post-hoc paired date-block intervals. Forward pilot selection improves mean skill over random transfer in all six families. Compact recurrent models achieve the lowest absolute mean errors, separating selection utility from forecasting quality. Retrospective five-candidate screening shows additional gains from farthest-first geometry before pilot ranking for MLP and Transformer. We distinguish across-target associations from within-target decisions and evaluate target heterogeneity, training seeds, and temporal-block sensitivity. Findings are exploratory for a fixed historical panel; the screening analysis measures no runtime saving.
BibTeX
@misc{verbii2026beyond,
  author = {Gordei Verbii and Georgii Killinkarov and Vladimir Yafarov},
  title = {{Beyond Wasserstein Proximity: Finding Better Transfer Sources}},
  year = {2026},
  eprint = {2610.35314},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2610.35314},
}
Figure