TL;DR
While Large Language Model (LLM) agents are increasingly deployed in quantitative trading, their heavy computational and token costs are rarely weighed against the actual value they add. This paper introduces TradeLens, a trace-grounded diagnostic toolkit that evaluates "agentic viability"—whether an agent's dynamic decisions generate enough incremental profit to pay for its own intelligence costs. Through extensive benchmarking, the authors reveal that many profitable trading agents actually lose money compared to simple passive baselines once LLM inference fees and execution frictions are factored in.
What Problem Does This Paper Address?
Existing evaluations of AI trading systems suffer from a profit-and-cost "double blind." Developers typically look at gross profit or standard risk-adjusted metrics like the Sharpe ratio. However, these metrics fail to answer two critical deployment questions:
- Where did the profit actually come from? A system might look highly profitable simply because the overall market went up (market beta) or because of a lucky initial asset choice, rather than the agent's dynamic trading skill.
- Is the agent economically viable? LLM agents incur substantial costs from continuous reasoning, tool usage, and API calls. If an agent makes \$630 in gross profit but costs \$250 to run, its net profit (\$380) might underperform a completely free, passive buy-and-hold strategy that makes \$450 over the same period.
The paper addresses this gap by shifting the evaluation paradigm from capability-centric performance ranking to trace-grounded economic diagnosis.
Methods
The authors propose TradeLens, an open-source audit toolkit that operates via two main layers:
- Accounting Layer: This layer decomposes gross profit and costs using counterfactual baselines.
- Profit Attribution: It breaks down returns into Market Effect (passive market movement), Asset Selection (value from the initial portfolio weights), and Timing Effect (incremental gains from the agent's dynamic reallocation decisions).
- Cost Attribution: It tracks LLM Inference Costs (tokens, retries), Trading Costs (commissions, slippage), Infrastructure Costs (data feeds, hosting), and Stochastic Overhead.
- Viability Metrics: It defines System Viability (does the overall system make a net profit?) and Agentic Viability (does the dynamic timing profit exceed the dynamic operational costs?).
- Diagnosis Layer: An LLM-powered agent analyzes the accounting traces to identify specific failure modes (e.g., aggressive trading cadence vs. weak predictive signals) and generates actionable strategy-level fixes.
Datasets
The framework was evaluated by instantiating it on top of a representative agentic trading system (AI-Trader).
- Asset Universe: A fixed portfolio of the 10 most liquid U.S. equities (including NVDA, AAPL, MSFT, AMZN, and TSLA).
- Market Windows: The primary evaluation used a two-month sideways market window (December 1, 2025, to January 30, 2026). The authors also conducted robustness checks across distinct market regimes (bearish and bullish) and an extended six-month evaluation (January 1 to July 1, 2026) comparing frontier models against the S&P 500 (SPY).
Key Insights and Ideas
- The "Viability Split": Models fall into distinct economic archetypes. For instance, Claude Sonnet 4.5 proved system-viable but agentically non-viable—it made money overall, but its dynamic trading decisions actually subtracted value relative to a static hold, failing to justify its token costs. Conversely, Qwen3-Max was agentically viable but system-non-viable because its active gains couldn't offset fixed infrastructure overhead.
- The High-Frequency Trap: Increasing decision frequency (e.g., moving from daily to hourly trading) consistently degraded viability. The issue wasn't just higher token costs; rather, more frequent decision points introduced additional trading noise and timing errors when the underlying predictive signal was weak.
- The Architecture Paradox: Cheaper or simpler reasoning architectures are not inherently better. A bare-bones Chain-of-Thought (CoT) approach had the lowest operational cost but yielded disastrous timing losses. More complex architectures like DeepFund, despite higher inference costs, achieved positive agentic viability by significantly improving dynamic reallocation quality.
Key Results
The authors benchmarked 10 flagship LLMs under identical configurations:
- Top Performers: GPT-5.2 and Mistral-Large-3 were the only models viable at both the system and agentic levels during the primary testing window. Mistral-Large-3 achieved the highest net agentic profit (\$833.52) due to superior timing execution.
- Failure Modes: DeepSeek-V3.2 was heavily penalized by poor asset selection (-\$1500.44), while GLM-4.7 and MiniMax-M2.1 suffered from massive negative timing effects (worse than -\$2000).
- Capital Scaling: Scaling initial capital from \$10k to \$500k did not automatically dilute costs to ensure success. For poorly performing models like DeepSeek-V3.2, larger capital scales simply amplified strategy-level timing losses, causing net losses to balloon at \$500k.
Limitations and Future Work
The authors note several limitations in their current setup:
- The experiments relied on controlled simulation environments with simplified execution-friction models; real-world market impact and finer-grained latency-to-fill slippage models need integration.
- The baseline comparison was restricted to a broad-market proxy (S&P 500) rather than highly optimized, non-LLM quantitative trading strategies.
- Certain cost parameters (like fixed infrastructure fees) require manual user configuration rather than automated tracking.
Future work aims to extend TradeLens to broader asset classes (like crypto or derivatives) and explore how multi-agent coordination or long-context retrieval affects the economic viability frontier.
Summary updated