Most machine-learning trading papers stop training when a validation score stops improving. That score almost always ignores the fees, spreads and price slippage a real strategy pays. Neural networks trained by gradient descent tend to fit smooth, slowly varying structure first and fast, jittery structure later. We therefore expect later checkpoints to trade more often: their positions flip in response to small wiggles in the predicted returns. If that holds, the stopping epoch works as an unacknowledged knob on trading frequency. A validation score that ignores costs will keep choosing later checkpoints that trade more, and the resulting backtest profits will disappear once each trade is charged.
This proposal measures that effect and asks whether choosing the stopping point on a cost-charged validation score, with a statistical test suited to noisy financial data, controls trading frequency as well as an explicit penalty in the training loss. The question has a second side. Picking a checkpoint on one historical validation window lets information from that window into the reported result. For language-model traders, that window may also be text the model memorized during pre-training. So the study treats checkpoint selection as a form of benchmark contamination, and it evaluates only on data the selection procedure never saw.
Core Claim
The claim is that in trading models, frictionless early stopping is a hidden source of inflated backtests, and that it acts through turnover (the fraction of the portfolio traded each period). Let $w_t$ be the portfolio weights at time $t$, $r_t$ the vector of asset returns, and $c$ a proportional cost per unit traded. The per-period net return is
$$r^{net}_t = w_{t-1}^\top r_t - c\,\lVert w_t - w_{t-1} \rVert_1 ,$$
and turnover is the time average of $\lVert w_t - w_{t-1}\rVert_1$. We make three testable predictions. First, turnover measured on held-out data rises with the training epoch $k$, which is what the spectral-ordering account of gradient descent implies. Second, the epoch that maximizes validation Sharpe ratio at $c=0$ comes later than the epoch that maximizes it at realistic $c$, and the gap grows with $c$. Third, stopping on net-of-cost validation gives most of the turnover reduction of an explicit temporal-consistency penalty, at no extra tuning cost. Only the first prediction is shared with generic regularization stories. The second and third are specific to this proposal and could fail.
Where This Sits
Four trading papers motivate the gap by conceding it. A Learnable Wavelet Transformer for Long-Short Equity Trading (WaveLSFormer) trains end to end on a return-and-Sharpe objective. It stops on standard epoch patience and states that its evaluation omits transaction costs, slippage and spreads. NumHTML predicts price moves from earnings-call text and audio, and its trading simulation likewise ignores fees. Learning to Trade Like an Expert fine-tunes Llama-3.1-8B into a trader with Q-LoRA, a low-cost fine-tuning method. It lists frictionless execution as a major limitation. Much earlier, A B-P ANN Commodity Trader observed that omitting frictions inflates trading utility. None of these papers asks whether their stopping rule interacts with the missing costs. That interaction is what we study.
Two papers constrain the design. Doubly Regularized Portfolio with Risk Minimization shows that small estimation errors produce extreme reallocations. It controls them with a sparsity penalty and a penalty on $\lVert w_t - w_{t-1}\rVert_2^2$, cutting turnover by 10.7% to 66.2% relative to gross-exposure-constrained portfolios. This is the explicit alternative our implicit mechanism must match. Its convex program does not carry over directly to neural trainers, but the same penalty can be added to a neural loss, so we test it head to head. Early stopping theory in random feature models derives exact test-error curves along gradient flow. Together with Bengio's practical recommendations, which treat early stopping as a regularizer, it supplies the reason to expect later checkpoints to fit higher-frequency structure. Neither is tested on outputs whose errors are charged per change.
The stopping method draws on sequential statistics. ScoreStop replaces patience, which has no interpretable scale, with a hypothesis test on whether further boosting rounds add value. Statistical Early Stopping for Reasoning Models uses conformal thresholds, which give finite-sample error guarantees, to halt generation, but it depends on the model voicing uncertainty. We borrow the testing idea rather than the uncertainty signal. Trading agents are documented to be confidently long-biased, as StockBench shows, so verbalized doubt is a poor trigger in this setting. Instance-dependent Early Stopping reports that its per-sample loss-acceleration signal breaks down under 40% label noise. This is a negative result: it tells us not to build on per-sample loss curvature for low signal-to-noise returns.
The contamination side has two sources. StockBench evaluates language-model agents on 2025 market data beyond their training cutoffs. It finds that most do not beat a 0.4% buy-and-hold return and that they collapse in downturns, which shows how much pre-cutoff results can flatter. Token-surprise analysis in Studying and improving reasoning in humans and machines shows that memorized items receive near-zero surprise, meaning the model predicts every token of them almost perfectly. We adapt that diagnostic to ask whether a validation window is familiar to the model being fine-tuned.
Research Program
The first component is a trajectory audit. For each model family, we save every checkpoint and compute on held-out data its gross return, its turnover, and its net Sharpe ratio across a sweep of proportional cost levels. The sweep runs from zero up to levels typical of liquid equities plus an allowance for half-spread and slippage, and is reported as a curve rather than a single assumed cost. The model families are a plain Transformer and a WaveLSFormer-style wavelet Transformer on hourly equities, a gradient-boosted tree baseline in the Qlib pipeline, and an ensemble reinforcement-learning agent in FinRL-Meta. From the audit we read off the first two predictions: the slope of turnover against epoch, and the displacement between the frictionless-optimal and net-optimal stopping epochs as a function of $c$.
The second component is the stopping rule. It targets two problems. Validation Sharpe ratios on financial data are noisy. And a single validation window can reward a checkpoint that happened to fit one market regime, which the research file flags as the "false plateau" failure of patience in non-stationary data. The rule therefore uses several disjoint validation windows, each separated from training and test by an embargo gap. At each epoch it runs a paired moving-block bootstrap on net returns, comparing the current checkpoint with the best so far. Training stops once the lower confidence bound on net-Sharpe improvement has stayed below zero across windows for a fixed number of epochs. The confidence level is the rule's one parameter. It is interpretable, unlike patience, and it is fixed in advance rather than tuned on test data.
The third component is a contamination arm, and it is deliberately narrow. We fine-tune Llama-3.1-8B on the Learning to Trade Like an Expert scenario data under two conditions. In one, checkpoints are selected on a pre-cutoff validation window. In the other, they are selected on a post-cutoff window. Test evaluation is the post-cutoff StockBench period in both cases, with costs charged. We also compute a per-window familiarity score: the model's average surprise on text serializations of that window's price and news records. We predict that pre-cutoff selection chooses later checkpoints and shows deceptively low turnover on the validation window, because memorized paths remove the whipsaw. We further predict that the turnover gap between validation and test is largest for windows with low surprise.
Baselines
The do-nothing reference is current practice: early stopping with epoch patience on frictionless validation loss or frictionless validation return, as in WaveLSFormer, reported net of costs at test. Alongside it we report buy-and-hold and equal-weight portfolios. On StockBench, buy-and-hold returned 0.4%, Kimi-K2 1.9% and Qwen3-235B-Think 2.5% (StockBench), all without spread or impact modeling. For the neural equity models, the reference numbers are WaveLSFormer's cumulative return on investment of 0.607 ± 0.045 and Sharpe of 2.157 ± 0.166, against 0.225 ± 0.056 and 1.024 ± 0.122 for a plain Transformer (WaveLSFormer), all frictionless. No reported net-of-cost result was found on Lacuna for WaveLSFormer, NumHTML, or the Learning to Trade Like an Expert agent, whose frictionless average return is 7.64%. Producing those numbers is itself a deliverable.
The naive fix is patience-based stopping on net-of-cost validation Sharpe over a single window. It is the cheapest thing a reviewer would propose, and it isolates whether the choice of metric alone carries the effect.
The strongest competitor is an explicit turnover penalty in the training loss. We add the Doubly Regularized Portfolio temporal-consistency term, $\lambda \lVert w_t - w_{t-1}\rVert_2^2$, and optionally an $L_1$ term on weight changes. We train to convergence with frictionless selection, and also with net-of-cost selection. The penalty weight $\lambda$ is tuned on the same validation windows, and the grid has as many settings as the number of stopping-rule configurations we try, so tuning budgets match. On the Fama–French 100 portfolios, the original paper reports a Sharpe ratio of 48.27% against 26.97% for equal weight, and the 10.7%–66.2% turnover reduction noted above (Doubly Regularized Portfolio). We rerun it rather than quote it, because those figures come from a convex solver without iterative training.
We also run inference-time position smoothing: an exponential moving average of the weights, with its decay tuned on validation. The validation-free weight-norm-peak stopping of Memorization Through the Lens of Sample Gradients is included as a label-free alternative.
Two ablations isolate the key ingredient. The first applies the bootstrap rule to the frictionless metric instead of the net metric. The second uses a single validation window instead of multiple embargoed windows. Every method uses identical data splits, architectures, seeds (at least five), and compute. The proposal beats the naive baseline if it achieves higher test net Sharpe at realistic $c$ with non-overlapping seed intervals. It matches the penalty baseline if its test net Sharpe lies within the penalty's interval while requiring no penalty tuning.
Why not just…?
Why not just add a turnover penalty to the loss instead of worrying about when to stop? This is the most serious threat. For proportional costs, the penalty is nearly differentiable and easy to add. If a tuned penalty with frictionless stopping clearly beats net-of-cost stopping, then the "implicit regularizer" story is wrong and the practical advice is simply to use the penalty. The experiments report that outcome plainly. Even then, the trajectory audit would show how much published frictionless-stopped results are inflated. We expect the two approaches to be complementary: a penalty tuned at one cost level may be mis-set at another, while stopping on net Sharpe adapts to whatever $c$ is charged at validation. The cost sweep tests exactly this.
Why not just smooth the positions after training? Smoothing lowers turnover, but it also delays the signal. If the late-epoch high-frequency components are noise, smoothing wastes the capacity spent fitting them, and a smoothed late checkpoint should do no better than an early one. If smoothing a late checkpoint does beat an early checkpoint, the late epochs were learning real fast signal, and our first prediction partly fails. The comparison therefore doubles as a test of the mechanism.
Why not just evaluate on post-cutoff data and forget contamination? For the test set we do exactly that. The contamination arm is about the validation window, which StockBench's forward-in-time fix does not address. A model can be tested cleanly on 2025 data yet have been selected on memorized 2023 data. If pre-cutoff and post-cutoff selection yield the same checkpoints and the same test results, the contamination concern is empirically moot for checkpoint selection, and we will say so.
Why not just use patience on net returns? That is the naive baseline. Our bootstrap-and-multiple-window rule has to earn its complexity against it. On noisy hourly and monthly returns, we expect patience to lock onto regime-specific bumps. The single-window ablation measures whether that happens.
Proposed Experiments
The equity arm uses five years of hourly U.S. equity bars, matching the span and 0.7:0.1:0.2 chronological split of WaveLSFormer as closely as a vendor feed allows. We report per-epoch gross return, turnover, and net Sharpe across the cost sweep. The portfolio arm reruns the Doubly Regularized Portfolio setting on FF25, FF48 and FF100, and on the 2008–2012 ETF, index and equity panels, with a neural allocator trained by gradient descent so that stopping has something to act on. The reinforcement-learning arm uses FinRL-Meta's stock environments and its reported 2020–2022 test window. The ensemble agent's 25.9% annual return and Sharpe of 1.53 (FinRL-Meta) serve as the frictionless reference. The language-model arm is described above, with evaluation on StockBench's post-cutoff period.
Three results decide the question. The first is the sign and size of the relationship between turnover and epoch. If turnover is flat, the core claim fails. The second is whether the gap between the frictionless-optimal and net-optimal epochs grows with $c$ across architectures. The third is the head-to-head against the tuned turnover penalty at matched tuning budget. A secondary check asks whether the familiarity score of the validation window predicts how far validation turnover underestimates test turnover in the language-model arm. Together these determine whether "when to stop" should be reported as part of a trading model's cost assumptions, or whether an explicit penalty makes the question irrelevant.
References
- A Learnable Wavelet Transformer for Long-Short Equity Trading and Risk-Adjusted Return Optimization
- NumHTML: Numeric-Oriented Hierarchical Transformer Model for Multi-Task Financial Forecasting
- Learning to Trade Like an Expert: Cognitive Fine-Tuning for Stable Financial Reasoning in Language Models
- A B-P ANN Commodity Trader
- Doubly Regularized Portfolio with Risk Minimization
- Model, sample, and epoch-wise descents: exact solution of gradient flow in the random feature model
- Practical Recommendations for Gradient-Based Training of Deep Architectures
- ScoreStop: Gradient-based early stopping using functional score tests
- Statistical Early Stopping for Reasoning Models
- StockBench:Can Llm Agents Trade Stocks Profitably In Real-world Markets?
- Instance-dependent Early Stopping
- Studying and improving reasoning in humans and machines
- Memorization Through the Lens of Sample Gradients
- FinRL-Meta: Market Environments and Benchmarks for Data-Driven Financial Reinforcement Learning