OutfitterBench: Evaluating LLM Agents on Long-Horizon Resource Allocation and Commitment Coordination

Jie Huang, Yuecheng Hong, Yao Chen, Shengding Jiao, Xudong Liu, Qiang YearXiv 2026 · arXiv

TL;DR

OutfitterBench is a new benchmark that evaluates LLM agents on their ability to manage a simulated outdoor rental business over a 200-day horizon. The study finds that while frontier models can perform basic tasks, most struggle with "commitment coordination"—the ability to keep future plans feasible while weighing the economic value of competing resource uses—often falling behind simple rule-based baselines.

What Problem Does This Paper Address?

As LLM agents move from simple chat interfaces to autonomous operators, they face a fundamental challenge: long-horizon resource allocation. In the real world, a decision made today (like accepting a rental booking for next week) creates a future constraint. An agent must not only ensure that the equipment is available and cleaned in time for that booking but also decide if that booking is more valuable than potential "walk-in" customers who might show up in the meantime.

Existing benchmarks often focus on short-term tool use or static planning. This paper addresses the gap in evaluating how agents maintain intertemporal feasibility (keeping successive decisions compatible) and intertemporal valuation (reassessing the value of resources as opportunities change) over hundreds of simulated steps.

Methods

The authors designed OutfitterBench, a simulation where an agent acts as a rental manager in Beijing.

  • The Task: Manage a shop for 200 days, handling ~1,100 reservation requests and uncertain walk-in demand.
  • The Constraints: A fixed capital ceiling for purchasing equipment, variable cleaning/repair times, and weather-dependent demand.
  • Agent Capabilities: Agents use 43 specialized tools to inspect assets, set prices, manage occupancy caps, and negotiate with customers.
  • Evaluation: Performance is measured by "Terminal Business Value"—the sum of cash and equipment resale value minus liabilities at the end of the season.

To look under the hood, the authors developed "process diagnostics" to track why agents fail. These include tracking "Pending-booking gaps" (instances where an agent lacks the gear to fulfill a promise) and "Action-induced new gaps" (when a new decision, like a walk-in rental, breaks a previous commitment).

Figure 2: The OutfitterBench rental shop combines two demand channels with a reusable equipment fleet. The agent manages bookings, walk-in prices and occupancy limits, and equipment investment using business records, public forecasts, and customer feedback. Calendar and weather jointly shape demand and the timing of equipment reuse, while a fixed equipment-capital ceiling constrains fleet expansion. Performance after 200 decision days and 14 automatic settlement days is terminal cash plus eligible equipment resale value, less outstanding liabilities.

Datasets

  • OutfitterBench: The primary contribution, featuring a 46-product catalog derived from real Taobao listings and historical Beijing weather data to drive realistic demand patterns.
  • Models Evaluated: Seven frontier LLM configurations, including GPT 6.1 Sol, Claude Opus 5.5, and DeepSeek V4.1 Flash.
  • Baseline: A rule-based "Calendar" policy that follows a fixed replenishment strategy and a simple availability check with a one-day cleaning buffer.

Key Insights and Ideas

  • The "Calendar" Hurdle: A simple, consistent rule-based policy (Calendar) outperformed five out of the seven LLM agents. This suggests that for long-horizon tasks, consistency and basic logic are often more valuable than the sophisticated but erratic reasoning of current LLMs.
  • Inconsistent Logic: Trajectory analysis revealed that agents often "knew" the rules (e.g., equipment needs one day for cleaning) but failed to apply them when schedules became crowded. Some agents would double-book assets even when the conflict was clearly visible in their records.
  • Volume $\neq$ Value: Some agents (like DeepSeek and Opus) served significantly more bookings than the top-performing agent (Sol) but ended up with lower business value. They often charged too little or incurred high cancellation penalties, failing to capture the true economic value of their limited inventory.
  • The Cost of Protection: Agents often struggle with "capacity protection." For example, an agent might block a tent for a future booking so early that they miss out on several profitable short-term rentals that could have been completed and cleaned before the reservation arrived.
Figure 3: Competing uses of a single tent. The new request overlaps a confirmed booking, whereas a walk-in rental could fit before it with a one-day margin after cleaning. Late return or servicing delays could eliminate this margin. Solid and dashed outlines denote confirmed and candidate rentals, respectively.

Key Results

  • Top Performer: Only GPT 6.1 Sol consistently outperformed the rule-based baseline, achieving a mean value creation of +2,556.78 CNY compared to the baseline's +1,119.51 CNY.
  • Reliability Gaps: Sol maintained a schedule reliability of ~96%, whereas lower-performing agents like GLM 5.2 dropped to ~73%, leading to massive compensation costs (over 1,500 CNY in some cases).
  • Capital Productivity: Sol generated significantly higher returns per unit of capital (1.38 CNY per 100 capital-days) than its closest competitors (~0.97 CNY), demonstrating superior resource valuation.
Figure 1: Business value created over the season, averaged equally over two world seeds for each agent. Curves use contract-adjusted economic value relative to the initial shop, accounting for equipment and outstanding obligations. Labels on the right report mean official net value creation (∆V ) at day 214. Shading marks the 14-day automatic settlement period. Calendar provides a fixed-policy reference between the highest- and lower-performing agents.

Limitations and Future Work

The authors acknowledge several limitations:

  • Rule-Based Customers: Currently, customer behavior follows fixed probabilistic models rather than being LLM-driven, which might miss some nuances of human negotiation.
  • Sample Size: The evaluation used only two world seeds per agent due to the high cost of long-horizon runs.
  • Tooling: Agents were restricted to a fixed set of business tools. Future work could explore "coding agents" that can write their own scripts to manage complex schedules.

OutfitterBench serves as a sobering reminder that while LLMs are excellent at following immediate instructions, "playing the long game" in resource management remains a significant frontier for AI research.

Summary updated

Abstract

In many long-horizon settings, an agent's own choices create future constraints: committing finite capacity establishes future obligations and forecloses later opportunities. Sustaining performance therefore requires both keeping evolving plans feasible and comparing the value of resource allocations with the alternatives they displace. Recent benchmarks increasingly evaluate LLM agents in continuing operations, but give less attention to how agents keep their own commitments feasible while reassessing resource allocations as opportunities change. We introduce OutfitterBench, where agents manage an outdoor-equipment rental business for 200 days, purchasing from a 46-product catalog and handling roughly 1,100 reservation requests. Under a fixed capital ceiling, reservations and uncertain walk-in demand compete for reusable equipment whose availability changes over time. Experiments with seven frontier LLMs show that several agents underperform a simple rule-based baseline, while only the leading agent consistently surpasses it. Trajectory analysis shows that some agents apply known constraints inconsistently and make new decisions that conflict with existing commitments. Agents with more reliable fulfillment can still secure low returns from comparable commitments or delay resource reallocation. The leading agent better connects preparation with economic choices but also misses opportunities for more profitable asset use. These findings highlight limitations in sustaining coherent resource plans and reassessing their value over time. The benchmark code is publicly available at https://github.com/adminblackjack/OutfitterBench.
BibTeX
@misc{huang2026outfitterbench,
  author = {Jie Huang and Yuecheng Hong and Yao Chen and Shengding Jiao and Xudong Liu and Qiang Ye},
  title = {{OutfitterBench: Evaluating LLM Agents on Long-Horizon Resource Allocation and Commitment Coordination}},
  year = {2026},
  eprint = {2611.17933},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.17933},
}
Figure