CBITS: Crypto BERT Incorporated Trading System

Gyeongmin Kim, Minsuk Kim, Byungchul Kim, Heuiseok LimIEEE Access 2023 · OpenReview

Abstract

Most textual analysis-based trading approaches in cryptocurrency (crypto) involve lexical, rule-based methods for extracting news sentiments. Furthermore, language models (LMs) are not always suitable for the crypto domain due to jargon that is not covered in general-purpose texts. This study answers the question of “Is it possible that the LMs can profit by effectively applying the sentiment score of the natural language processing task with chart score in the BTC trading system?” by focusing on the effectiveness of both scores, which significantly affect the profit of the trading system. We introduce CBITS: Cryptocurrency BERT Incorporated Trading System based on pre-trained LMs for Korean crypto sentiment analysis to aid Bitcoin (BTC) trading models. We pre-trained crypto-specific LMs, which are transformer encoder-based architectures. Along with our pre-trained LMs, we also present our custom fine-tuning dataset used to train our LMs on the BTC sentiment classifier and show that using sentiment scores along with BTC chart data boosts the performance of BTC trading models and also allows us to create a market-neutral trading strategy.

Introduction

Predicting Bitcoin (BTC) prices is notoriously difficult due to extreme volatility and the heavy influence of "hype" and news. While sentiment analysis is a common tool in traditional finance, it faces significant hurdles in the cryptocurrency market. General-purpose language models (LMs) often struggle with crypto-specific jargon like "HODL" or "gas fees." Furthermore, the Korean language's agglutinative nature—where complex words are formed by stringing morphemes together—makes standard tokenization less effective.

Existing crypto sentiment tools often rely on simple rule-based methods that miss the nuanced context of financial news. CBITS (Crypto BERT Incorporated Trading System) addresses these gaps by fusing domain-specific Korean NLP with technical chart analysis to create a robust, profitable trading framework.

System Architecture

The researchers developed a multi-stage pipeline to create CBITS, integrating natural language processing with sequential data modeling. The process begins with domain-specific pre-training of transformer architectures—BERT, RoBERTa, and DeBERTa—using a massive Korean crypto corpus. To handle the 512-token limit of BERT models, the system employs KR-SBERT to perform semantic searches, extracting the most relevant sentences from long news articles to preserve context.

The distribution of target classes shows the frequency of significant price rises (0), drops (1), and neutral movements (2) used to train the prediction model.

The final trading decisions are handled by a hybrid model. While the researchers evaluated LSTM and XGBoost, TabNet proved to be the most effective. TabNet uses sequential attention to mimic decision trees, making it ideal for processing the combination of numerical technical indicators (like VWAP and Chaikin Money Flow) and the categorical sentiment scores generated by the LMs.

Data and Labeling

The system was trained on a diverse set of data sources:

  • Pre-training Corpus: 900,000 Korean texts collected from news sites, blogs, and forums.
  • Gold & Silver Datasets: A high-quality dataset manually labeled by professional day traders, supplemented by a larger "silver" dataset labeled by the best-performing model and human-verified to mitigate class imbalance.
  • Chart Data: 4-hour interval BTC/USDT trading data from Binance (2017–2022).
The distribution of target classes shows the frequency of significant price rises (0), drops (1), and neutral movements (2) used to train the prediction model.

The model predicts three distinct classes: price increases of $\geq 0.75\%$ (Class 0), price decreases of $\leq -0.75\%$ (Class 1), and neutral movements (Class 2). This classification allows for a market-neutral strategy, enabling the bot to profit in both bullish and bearish environments.

Key Results and Insights

The integration of domain-specific sentiment analysis provided a clear competitive advantage:

  • The "Crypto-Speak" Advantage: The domain-adapted CryptoRoBERTa outperformed general multilingual models by roughly 3 points in F1-score, confirming that general models are often confused by crypto-specific terminology.
  • Sentiment Importance: Feature importance analysis revealed that negative and positive sentiment scores were the most influential factors in the model's decision-making, ranking higher than volume changes or minor price fluctuations.
  • Backtesting and Live Performance: In backtesting, the TabNet + CryptoRoBERTa model achieved a cumulative profit of 304.65%, whereas a "Buy and Hold" strategy resulted in a 5.92% loss. During a live validation run in mid-2022, CBITS maintained a +7.926% profit with a minimal maximum drawdown of -1.274%.

Conclusion

CBITS demonstrates that combining domain-specific NLP with technical analysis significantly boosts trading performance. While the current model uses a rigid exit strategy (0.75% take-profit), future work aims to incorporate on-chain data and social media signals to further refine sentiment detection. Moving toward multi-modal structures that process text and chart images simultaneously could represent the next evolution in automated cryptocurrency trading.

Summary updated

Abstract

Most textual analysis-based trading approaches in cryptocurrency (crypto) involve lexical, rule-based methods for extracting news sentiments. Furthermore, language models (LMs) are not always suitable for the crypto domain due to jargon that is not covered in general-purpose texts. This study answers the question of “Is it possible that the LMs can profit by effectively applying the sentiment score of the natural language processing task with chart score in the BTC trading system?” by focusing on the effectiveness of both scores, which significantly affect the profit of the trading system. We introduce CBITS: Cryptocurrency BERT Incorporated Trading System based on pre-trained LMs for Korean crypto sentiment analysis to aid Bitcoin (BTC) trading models. We pre-trained crypto-specific LMs, which are transformer encoder-based architectures. Along with our pre-trained LMs, we also present our custom fine-tuning dataset used to train our LMs on the BTC sentiment classifier and show that using sentiment scores along with BTC chart data boosts the performance of BTC trading models and also allows us to create a market-neutral trading strategy.
BibTeX
@inproceedings{kim2023cbits,
  author = {Gyeongmin Kim and Minsuk Kim and Byungchul Kim and Heuiseok Lim},
  title = {{CBITS: Crypto BERT Incorporated Trading System}},
  booktitle = {IEEE Access},
  year = {2023},
  url = {https://openreview.net/forum?id=bSbgGFBBvf},
}
Figure