DDSL: Iterative Self-Labeling for Financial Market States

Xianbin Gu, Qingyue Zhu, Yuhong GengarXiv 2026 · DOI · arXiv

TL;DR

DDSL (Data-Driven Self-Labeling) is a novel framework that iteratively refines financial market state labels by coupling a structured Viterbi labeler with a Transformer classifier. By feeding model predictions back into the labeling process, it produces market states that are significantly more "learnable" (78%–96% F1 improvement) while maintaining the underlying price structure and temporal logic.

What Problem Does This Paper Address?

In most machine learning tasks, we have a clear "ground truth" (e.g., "this is a cat"). In financial markets, "states" like Bull or Bear are human constructs with no unique, universally accepted definition.

Traditionally, researchers use rule-based methods (like price crossovers) or unsupervised segmentation (like HMMs) to create labels. However, these labels are often defined independently of the models meant to predict them. This creates a "learnability gap": a labeling system might perfectly describe historical price action but be impossible for a classifier to recover using only the information available at prediction time. DDSL aims to bridge this gap by making the labeling process aware of what the classifier can actually learn.

Methods

DDSL is a closed-loop, iterative framework. Its core components are:

  1. A 9-State Joint Representation: Instead of simple 3-class labels (Bear/Neutral/Bull), DDSL uses a $3 \times 3$ grid that jointly encodes the current trend and the future trend (e.g., BULL_TO_BEAR).
  2. The Structured Viterbi Labeler: This component assigns labels by minimizing an "energy" function. This function balances three things:
  3. Price Emissions: How well the label matches price statistics (slope, volatility).
  4. Temporal Grammar: A set of soft constraints ensuring that the "future" part of today's label matches the "current" part of tomorrow's label.
  5. Model Feedback ($\beta_{model}$): This is the "secret sauce." The log-probabilities from a Transformer classifier are injected into the Viterbi objective, nudging state boundaries toward configurations the model finds easier to distinguish.
  6. The Iterative Loop: The system starts with pure price-based labels (Iteration 0). It then alternates between training the Transformer on the current labels and using that Transformer's feedback to generate a refined set of labels for the next round.
Figure 1: DDSL label evolution at selected iterations for stock 600760 (Shanghai Stock Exchange, 2000/03/28–2000/11/08). Segment boundaries shift under model feedback, with progressively smaller changes at later iterations.

Datasets

The authors evaluated DDSL across four diverse equity datasets covering two major markets:

  • Chinese A-shares: SSE50 (large-cap), CSI300 (broad market), and CSI500 (mid/small-cap).
  • US Market: NASDAQ100. The observation period spanned from January 2000 to December 2025, providing a robust test bed across different market regimes and capitalizations.

Key Insights and Ideas

  • The Learnability-Discriminability Tradeoff: This is the paper's most critical finding. Initially, model feedback improves both the model's accuracy (learnability) and the geometric separation of price features (discriminability). However, after the first iteration, further refinement continues to increase learnability at the expense of discriminability. The labels become "too" aligned with the model's internal representations, potentially losing some raw economic signal.
  • Cross-Architecture Transfer: To prove the gains weren't just the Transformer "overfitting" its own feedback, the authors used an independent GRU classifier as a probe. The GRU—which never participated in the labeling loop—showed nearly identical performance gains, suggesting DDSL uncovers objectively more recoverable market structures.
  • Stability vs. Collapse: Unlike some self-labeling schemes that collapse into a single dominant class, DDSL's grammar and price constraints keep the label distribution stable. Entropy and class coverage remained consistent throughout the refinement process.

Key Results

  • Massive Learnability Gains: Across all four datasets, the held-out joint-state macro F1 score increased by 78% to 96% compared to the initial dynamic-programming baseline.
  • Convergence: Most of the gains occur in the first two iterations. By iteration 5, the label systems stabilize, with only 3%–5% of labels changing between rounds.
  • External Validation: In a non-anticipative event study, the refined labels strengthened a conditional BEAR $\rightarrow$ BULL reversal signal on the Chinese datasets. Interestingly, the effect was reversed on the NASDAQ100, highlighting that while DDSL improves learnability, the underlying economic content remains market-dependent.

Limitations and Future Work

The authors acknowledge several areas for further exploration:

  • Model Dependence: While the GRU probe helps, the test-split labels still technically depend on the model's outputs from the previous iteration. Future work could involve withholding a completely "clean" time period from the refinement loop.
  • Fixed Snapshots: The study used a fixed 2025 index constituent list rather than point-in-time membership, which could introduce survivorship bias if used for backtesting trading strategies.
  • Adaptive Schedules: Future research could develop "early stopping" criteria for the labeling loop to find the optimal balance on the learnability-discriminability map automatically.

Summary updated

Abstract

Financial market states lack a unique ground truth and must therefore be constructed from price data. Conventional labeling methods typically define states independently of the downstream classifier, leaving unresolved whether the resulting labels can be recovered from information available at prediction time. We propose DDSL, an iterative self-labeling framework that couples a price- and grammar-guided Viterbi labeler with a Transformer classifier through model feedback. DDSL constructs a nine-state representation that jointly encodes current and future market trends while retaining structured price emissions and temporal constraints throughout refinement. Experiments on SSE50, CSI300, CSI500, and NASDAQ100 show that DDSL increases held-out joint-state macro F1 by 78%-96% over single-pass dynamic-programming labeling. An independent GRU classifier that never participates in the labeling loop reproduces the same trend, while label entropy and class coverage remain stable, indicating that the gain transfers across architectures and is not caused by label collapse. We further show that learnability is controlled by both iteration and the model-feedback weight $β_{\mathrm{model}}$. Iterative gains diminish as successive label systems converge, whereas stronger feedback increasingly widens the performance gap between the feedback Transformer and the independent GRU. Increased learnability is accompanied by declining segment-level price-feature separation after an initial improvement. Finally, a non-anticipative event study provides partial external validation: refinement strengthens a conditional BEAR$\rightarrow$BULL reversal signal on the three A-share datasets, while the corresponding effect has the opposite sign on NASDAQ100. These results position DDSL as a structured framework for constructing market-state labels whose learnability can be explicitly controlled and evaluated.
BibTeX
@misc{gu2026ddsl,
  author = {Xianbin Gu and Qingyue Zhu and Yuhong Geng},
  title = {{DDSL: Iterative Self-Labeling for Financial Market States}},
  year = {2026},
  eprint = {2611.22958},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.22958},
}
Figure