TL;DR
This paper tackles the debate around high-frequency trading (HFT) profitability by simulating an "omniscient" trader to establish an upper bound on potential profits. Surprisingly, even with perfect foresight, the maximum achievable profits for aggressive HFT strategies are modest, highlighting a fundamental tension between trading costs and short holding periods.
What Problem Does This Paper Address?
The rise of high-frequency trading (HFT) has sparked significant public, media, and regulatory concern. The core fear is that sophisticated trading firms, leveraging advanced technology and speed, can exploit market inefficiencies and less sophisticated traders. However, objective, large-scale empirical studies on the actual profitability of HFT have been scarce, leading to a debate often fueled by anecdotal evidence and speculation. This paper aims to provide a rigorous, data-driven perspective by quantifying the maximum possible profitability of aggressive HFT strategies, thereby offering a controlled empirical benchmark.
Methods
The authors introduce the Omniscient Trader Methodology (OTM). This novel approach simulates a hypothetical trader with perfect knowledge of future price movements. By applying this OTM to historical market data, the researchers can identify all potentially profitable trades and calculate their maximum possible profit. Crucially, to align with the concept of HFT, the simulation imposes constraints:
- Aggressive Order Placement: Only market orders are used, meaning the trader must pay the bid-ask spread and incur execution costs. This contrasts with passive HFT, which provides liquidity.
- Short Holding Periods: Trades are held for very short durations, ranging from 10 milliseconds to 10 seconds. This reflects the core idea of HFT capitalizing on fleeting opportunities.
The OTM is applied in two stages: first, using high-resolution order book data for a select set of liquid stocks, and then extrapolating to a broader market using less granular Trade and Quote (TAQ) data and regression models. The methodology is designed to err on the side of overestimation to ensure the results represent an upper bound.
Datasets
The study utilizes two primary datasets:
- Order Book Data: Complete event information (order placements, cancellations, modifications, and trade executions) for 19 highly liquid NASDAQ stocks throughout 2008. This data allows for the full reconstruction of historical order books.
- Trade and Quote (TAQ) Data: A broader dataset providing bid/ask prices and available liquidity for a much larger universe of US stocks and exchanges. This data is used for market-wide extrapolation.
Key Insights and Ideas
- The Horizon-Cost Trade-off is Paramount: The most significant finding is the strong tension between the short time horizons of HFT and the associated trading costs (bid-ask spreads). To be profitable, trades must be held long enough for price movements to overcome these costs, but HFT's advantage diminishes rapidly as holding periods lengthen.
- Aggressive HFT is the Focus: The paper argues that aggressive order placement (using market orders) is the primary driver of potential concern regarding HFT, as it directly incurs costs and interacts with the market in a way that can impact prices. Passive HFT, which provides liquidity, is deemed less of a concern.
- "Omniscience" Still Yields Modest Profits: Even with perfect foresight and the ability to execute optimal trades, the total potential profits for aggressive HFT are surprisingly limited, especially at very short holding periods. This suggests that the market is more efficient than often assumed, or that the costs are a more significant barrier than speed alone.
- Liquidity is Key to Profitability: The study reveals that profitability is highly concentrated in a small number of the most liquid stocks. This indicates that the depth of the order book (available shares at competitive prices) is a critical factor for HFT success.
- Crisis Spike is Real but Limited: HFT profits saw a notable spike during the 2008 financial crisis, but even then, the upper bounds on profitability remained relatively modest compared to overall market volume.
Key Results
- Total Profitability Upper Bound: For the 19 NASDAQ stocks studied, the estimated maximum total profitability for aggressive HFT in 2008 was $3.4 billion at a 10-second holding period, falling to just $62,000 at a 10-millisecond holding period.
- Market-Wide Extrapolation: Extrapolating to the entire US equity market, the estimated upper bound on total aggressive HFT profits for 2008 was approximately $21 billion. This represents less than 0.05% of the total annual trading volume.
- Concentration of Profits: The top 5 most liquid stocks accounted for 73% of the total estimated profits among the 19 studied stocks.
- Impact of Holding Period: Profitability decays sharply as holding periods decrease. For instance, at a 1-second holding period, total profitability was only about 3.6% of that at a 10-second holding period.
- Composite vs. Primary Exchange: Including trading across multiple exchanges (composite pricing) increased estimated profits by an average factor of 2.1 compared to trading only on the primary exchange (NASDAQ).
Limitations and Future Work
The authors acknowledge several limitations:
- Discrete Holding Periods: The study uses fixed holding periods. Additional profit opportunities might exist at intermediate values, though preliminary tests suggest only a modest increase in profits.
- No Mixed Order Types: The simulation does not allow for entering with a market order and exiting with a limit order (or vice versa).
- No Market Impact: The simulation assumes the OT's trades do not influence market prices or liquidity, which is unrealistic and leads to overestimation.
- No Manipulation: The study does not account for potential market manipulation strategies.
- Focus on Aggressive HFT: The findings are limited to aggressive HFT and do not address the profitability of passive, liquidity-providing HFT.
Future work could explore these limitations, including simulating passive HFT, incorporating market impact, and examining a wider range of trading strategies and market conditions. The authors also note that 2009 profits were estimated to be about half of 2008 profits, suggesting a potential downward trend.
Summary updated