Modeling Market Frictions in Algorithmic Trading

116 concepts115 papers2 proposals

Imagine developing a machine learning model that predicts short-term stock movements with 60% accuracy. In a simulated environment, this model looks like a highly profitable strategy. However, when deployed in a live market, the strategy rapidly loses money.

This disconnect is one of the most pervasive challenges in financial engineering. It occurs because standard machine learning evaluation metrics often treat the stock market as a frictionless environment—a mathematical ideal where assets can be bought and sold instantly, in any quantity, at historical mid-prices, and without overhead. In reality, financial markets are highly viscous. Every trade incurs costs that act like physical friction, quietly eroding profit margins until a theoretically sound model becomes practically non-viable.

The Illusion of Frictionless Markets

When evaluating predictive models, researchers frequently equate directional accuracy (e.g., predicting whether a price will go up or down) with financial profitability. As noted as early as the work by Collard, excluding transaction costs like brokerage commissions and market slippage consistently leads to an overestimation of a trading model's real-world utility.

This overestimation is particularly severe in modern deep learning applications. Whether a model relies on character-based language models to parse financial news (dos Santos Pinheiro et al.) or extracts structured event embeddings from headlines (Ding et al.), the resulting trading signals are only as good as the simulation environment used to test them. If a simulation assumes that a fund can rebalance its portfolio instantly and costlessly, it creates an illusion of alpha that vanishes under real-world execution constraints (Li et al.).

The Anatomy of Market Frictions

To understand why these models fail, we must break down "transaction costs" into three distinct layers of friction:

  1. Direct Operational Costs: These are the explicit fees paid to intermediaries, including brokerage commissions, exchange fees, and regulatory taxes. While small per trade, they compound rapidly for high-frequency or high-turnover strategies (Abernethy et al.; Li et al.).
  2. Execution Slippage: Backtests often evaluate performance using daily average prices or historical closing prices (Ye et al.). In live trading, however, there is a delay between a model generating a signal and the order executing. In volatile markets, the price can move unfavorably during this window, meaning the realized execution price rarely matches the theoretical backtest price.
  3. Market Impact and Liquidity Constraints: In an idealized backtest, you can buy or sell an infinite number of shares at the current price. In reality, order books have finite depth. Executing a large order requires consuming multiple price levels (the bid-ask spread), which actively pushes the market price against you (Tan et al.; Liu et al.).
[Model Signal Generated] 
       │
       ▼
[Direct Costs] ──► Brokerage Fees & Taxes (Fixed/Percentage drain)
       │
       ▼
[Slippage]     ──► Execution Delay (Price moves before order fills)
       │
       ▼
[Market Impact]──► Order Book Depth (Large volume pushes price unfavorably)
       │
       ▼
[Realized Profit/Loss] ◄── Often significantly lower than simulated returns

Why Frictions are Omitted: The Complexity Penalty

If market frictions are so destructive, why do many financial forecasting papers continue to omit them? The primary reason is mathematical and computational convenience.

As Shen et al. point out, incorporating realistic market frictions introduces severe mathematical complexity. Classical portfolio optimization frameworks, such as the foundational mean-variance theory, rely on clean, differentiable, and convex objective functions (Boyd et al.). Adding non-linear, path-dependent transaction costs or variable fee structures breaks these elegant mathematical assumptions, making the optimization problem significantly harder to solve analytically (Edmond et al.; He et al.).

Consequently, researchers frequently default to idealized market assumptions to keep their optimization algorithms tractable, even if it means sacrificing real-world validity (Neufeld et al.; Wu et al.).

Designing Friction-Aware Models

To bridge the gap between simulation and reality, the research direction has shifted toward embedding friction directly into the model design and evaluation phases. Rather than treating transaction costs as an afterthought during post-hoc analysis, modern frameworks use several key mechanisms to mitigate their impact:

Turnover Regularization

One approach is to penalize the model for trading too frequently. By adding mathematical penalties directly to the objective function, models learn to balance the expected return of a trade against the cost of executing it. For instance, the doubly regularized framework proposed by Shen et al. enforces both sparsity (holding fewer assets) and consistency (minimizing changes between trades), drastically slashing the turnover that typically erodes algorithmic profits.

Selective Execution and "Soft" Decisions

Another intuitive solution is to teach models when not to trade. In high-stakes environments, a forced binary decision (always buy or always sell) forces execution even when confidence is low, racking up unnecessary transaction fees.

By utilizing "soft" classification frameworks, models can choose to remain undecided in ambiguous market regions (Baram). Similarly, frameworks like LARA focus exclusively on highly predictable "opportunity windows" (Zeng et al.), ensuring that a trade is only executed when the predicted alpha comfortably exceeds the expected cost of entry and exit.

Realistic Backtesting Suites

Finally, evaluating models requires moving away from simple directional accuracy metrics toward comprehensive execution suites. Modern benchmarks increasingly reject single-share trading constraints and flat-fee assumptions (Yang et al.). By forcing models to interact with simulated order books, variable fee structures, and liquidity constraints, researchers can ensure that an algorithm's predictive power actually translates to sustainable, real-world wealth accumulation.

Key Foundational Work

A B-P ANN Commodity Trader (1990) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_omission_of_transaction_costs_d805b4e7,
  title = {omission of transaction costs},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/omission-of-transaction-costs/rdr_d805b4e7eabd44c087aad14b8a9c10a9},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_omission_of_transaction_costs_d805b4e7_v1,
  title = {omission of transaction costs},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/omission-of-transaction-costs/rdr_d805b4e7eabd44c087aad14b8a9c10a9/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake