Meta-Learning Reinforcement Learning for Crypto-Return Prediction

Junqiao Wang, Zhaoyang Guan, Guanyu Liu, Tianze Xia, Xianzhi Li, Shuo Yin, Xinyuan Song, Chuhan Cheng, Tianyu Shi, Alex Lee

Abstract: Predicting cryptocurrency returns is notoriously difficult: price movements are driven by a fast-shifting blend of on-chain activity, news flow, and social sentiment, while labeled training data are scarce and expensive. In this paper, we present Meta-RL-Crypto, a unified transformer-based architecture that unifies meta-learning and reinforcement learning (RL) to create a fully self-improving trading agent. Starting from a vanilla instruction-tuned LLM, the agent iteratively alternates between three roles—actor, judge, and meta-judge—in a closed-loop architecture. This learning process requires no additional human supervision. It can leverage multimodal market inputs and internal preference feedback. The agent in the system continuously refines both the trading policy and evaluation criteria. Experiments across diverse market regimes demonstrate that Meta-RL-Crypto shows good performance on the technical indicators of the real market and outperforming other LLM-based baselines.

1. Introduction

Predicting cryptocurrency prices is a "wicked" problem. Unlike traditional equities, crypto is driven by a volatile mix of on-chain activity (like gas fees and wallet movements), rapid-fire news cycles, and social media sentiment. Current AI approaches face two major hurdles: data scarcity, as high-quality human-labeled financial data is expensive, and reward hacking, where simple reinforcement learning (RL) models chase high returns while ignoring massive risks or falling prey to linguistic biases.

2. System Architecture

The core innovation of Meta-RL-Crypto is a triple-loop learning process where a single Large Language Model (specifically a fine-tuned Llama-7B) cyclically adopts three distinct roles to refine its trading policy without human intervention.

Overview of the Meta-RL-Crypto architecture, showing data integration and the self-improvement cycle.
  • The Actor: Acts as the trader, processing multimodal inputs—on-chain data, news digests, and sentiment snapshots—to generate price forecasts and rationales.
  • The Judge: Evaluates the Actor’s performance using a multi-objective reward vector including absolute returns, Sharpe ratio, drawdown control, and sentiment alignment.
  • The Meta-Judge: Refines the Judge’s reward policy using preference comparisons (Direct Preference Optimization) to ensure evaluation criteria remain robust against biases.

3. Data and Experimental Setup

The researchers developed a comprehensive multimodal pipeline sourcing on-chain data from CoinMarketCap and Dune Analytics, and off-chain news from outlets like Bloomberg and Reuters via the GNews API. The model was tested on Bitcoin (BTC), Ethereum (ETH), and Solana (SOL) across three distinct market regimes: Bullish, Bearish, and Sideways.

4. Results and Discussion

Meta-RL-Crypto was evaluated in a backtest with a $1,000,000 starting portfolio against several state-of-the-art LLM baselines.

Comparative performance of Meta-RL-Crypto against LLM baselines across different market regimes.

As shown in the performance data, the model significantly outperformed baselines in challenging bear market conditions, limiting losses to -8.00% compared to GPT-4's -22.00%. It also achieved the highest Sharpe Ratio (0.30) in bull markets. Beyond quantitative metrics, professional crypto analysts rated Meta-RL-Crypto higher than GPT-4 for the soundness and relevance of its trading rationales.

5. Conclusion

Meta-RL-Crypto demonstrates that a self-improving, triple-loop architecture can effectively navigate the complexities of cryptocurrency markets. By integrating multimodal "common sense" and fighting reward hacking through multi-objective optimization, the framework provides a robust path forward for autonomous financial agents. Future work will explore expanding asset diversity to lower-liquidity markets and incorporating more granular data streams.

Summary updated

Abstract

Predicting cryptocurrency returns is notoriously difficult: price movements are driven by a fast-shifting blend of on-chain activity, news flow, and social sentiment, while labeled training data are scarce and expensive. In this paper, we present Meta-RL-Crypto, a unified transformer-based architecture that unifies meta-learning and reinforcement learning (RL) to create a fully self-improving trading agent. Starting from a vanilla instruction-tuned LLM, the agent iteratively alternates between three roles-actor, judge, and meta-judge-in a closed-loop architecture. This learning process requires no additional human supervision. It can leverage multimodal market inputs and internal preference feedback. The agent in the system continuously refines both the trading policy and evaluation criteria. Experiments across diverse market regimes demonstrate that Meta-RL-Crypto shows good performance on the technical indicators of the real market and outperforming other LLM-based baselines.
BibTeX
@misc{wang2025meta,
  author = {Junqiao Wang and Zhaoyang Guan and Guanyu Liu and Tianze Xia and Xianzhi Li and Shuo Yin and Xinyuan Song and Chuhan Cheng and Tianyu Shi and Alex Lee},
  title = {{Meta-Learning Reinforcement Learning for Crypto-Return Prediction}},
  year = {2025},
  eprint = {2509.09751},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2509.09751},
}
Figure