Designing adaptive trading agents

David Pardoe, Peter StoneSIGecom Exch 2011 · DOI

TL;DR

This paper tackles the challenge of designing autonomous trading agents that can adapt to new markets, even with limited prior experience. They propose and evaluate novel machine learning algorithms, particularly focusing on transfer learning, to enable agents to leverage past market interactions (real or simulated) to improve performance in unfamiliar environments.

What Problem Does This Paper Address?

The rise of electronic commerce has spurred interest in autonomous trading agents. A key challenge is designing agents that can effectively interact with other market participants (human or automated). Traditional approaches often rely on game theory to find equilibria or use historical data to build market models. However, these methods falter when historical data is scarce or participant preferences are unknown, such as when an agent enters a completely new market. This paper addresses the problem of an agent entering a novel market with no prior experience, exploring how it can adapt by learning online and, crucially, by leveraging experience from similar past markets or simulations.

Methods

The authors frame the problem around four core questions: what the agent needs to learn, what prior experience is available, what learning method to use, and how to leverage prior experience. They propose four general categories of adaptation methods:

  • A. Learn from the new market only: Standard online learning.
  • B. Identify the new market: Attempt to classify the new market from a space of possibilities, using prior experience to inform this space.
  • C. Tune learning parameters: Use prior experience to tune the parameters or structure of a learning model, which then learns solely from the new market.
  • D. Combine data/models: Learn from both new and prior market data/models, with the challenge being to identify relevant sources and combine them effectively.

The paper introduces and experimentally validates novel algorithms within these categories, with a particular focus on regression transfer algorithms for price prediction. They identified a boosting-based regression transfer algorithm as a top performer.

Datasets

The paper discusses experiments conducted within the context of the Trading Agent Competition (TAC). Specifically, they mention:

  • Supply Chain Management scenario: Six agents compete as computer manufacturers.
  • Ad Auctions scenario: Eight agents compete as advertisers bidding in keyword auctions.

The agents' performance is evaluated based on their success in these simulated market environments. The authors also note that data from previous TAC competitions (different sets of agents) is used as "previous market experience" for transfer learning.

Key Insights and Ideas

  • Leveraging past experience is crucial: Agents don't have to learn from scratch; prior experience in similar markets or simulations can significantly accelerate adaptation and improve performance.
  • Transfer learning is key for novel markets: When historical data for the current market is limited, transfer learning from related markets becomes a powerful strategy.
  • Robustness matters: Agents need to adapt quickly to new situations but also remain robust to unexpected behaviors from other participants.
  • Price prediction accuracy is paramount: In scenarios like TAC auctions, the ability to accurately predict market prices is directly linked to agent performance.
  • Boosting-based transfer learning shows promise: A specific boosting-based regression transfer algorithm was found to be a consistent top performer across diverse datasets.

Key Results

The paper highlights the success of their agent, TacTex, which was the reigning champion in both the TAC supply chain management and ad auction scenarios in 2010. The core contribution is the development and experimental validation of novel algorithms for market adaptation. While specific quantitative results like percentage improvements are not detailed in this extended abstract, the champion status of their agent strongly suggests superior performance achieved through their adaptive strategies. They emphasize that their methods allow agents to quickly identify optimal parameters when bidder behavior is similar to past experience, while remaining robust to unexpected situations.

Limitations and Future Work

The authors acknowledge that the effectiveness of their methods depends on the similarity between past and new markets. They also note that the specific answers to the four core questions (what to learn, what experience is available, what learning method, how to use experience) are dependent on the particular market scenario.

Future work is implied by the exploration of different categories of adaptation and the identification of a top-performing algorithm. The paper suggests that their work is broadly applicable to anyone interested in applying machine learning to agent-based automated trading, including those participating in competitions like TAC, and points to upcoming scenarios like the electric power market.

Summary updated

BibTeX
@article{pardoe2011designing,
  author = {David Pardoe and Peter Stone},
  title = {{Designing adaptive trading agents}},
  journal = {SIGecom Exch.},
  year = {2011},
  doi = {10.1145/1998549.1998557},
}
Figure