Introduction
In behavioral finance, the framing effect describes a cognitive bias where individuals make different decisions based on how information is presented rather than the objective facts. This bias is a significant hurdle for human investors and, as recent research shows, for Large Language Models (LLMs) like ChatGPT. When tasked with financial sentiment analysis, LLMs often mirror human reactive tendencies, focusing on short-term sensationalism rather than long-term economic indicators.
This paper introduces the Classify-and-Rethink (CAR) framework, a multi-step prompting strategy designed to mitigate these biases. By structuring the reasoning process, the CAR framework enables LLMs to function as more rational, long-term investors, significantly improving performance in gold market trading.
The Classify-and-Rethink (CAR) Framework
The CAR framework moves away from simple "one-step" sentiment scoring. Instead, it employs a zero-shot reasoning chain that mimics professional financial analysis:
- Classify: The model first categorizes news into one of six domains (e.g., Geopolitical events, Macroeconomic data, Central Bank policy). This step forces the model to identify the underlying nature of the information before judging its impact.
- Initial Scoring: Based on the identified category, the model provides an initial bullish or bearish score ranging from -1 to 1.
- Rethink: This critical step prompts the model to adopt a "reverse thinking perspective." It reflects on whether the initial score is reasonable within a long-term trend context, making necessary adjustments to produce a final, balanced score.
Following the scoring process, a trading strategy is applied by calculating a moving average of the scores from the 20 most recent news articles. If the average crosses a specific threshold, it triggers a "Buy" (full position) or "Sell" (empty position) signal.
Experimental Setup and Data
The study evaluated the CAR framework using data from January 2, 2018, to June 13, 2023. The dataset included:
- Price Data: Daily closing prices of the Shanghai Gold Exchange spot gold index (Au9999.SGE).
- News Data: Chinese-language gold market news from dyhjw.com.
Traditional sentiment models often struggle with this news data, frequently labeling it as "neutral" due to a lack of overt emotional language. This makes it an ideal testbed for the sophisticated reasoning capabilities of LLMs.
Key Insights: The Logic of Distribution
A primary finding of the study was the impact of the CAR framework on the distribution of sentiment scores. Simple "One-Step" prompts often resulted in heavily skewed distributions. While the "Classify" step alone tended to push scores toward extremes, the full CAR strategy produced a near-normal distribution. This distribution is more representative of actual market dynamics, where extreme events are rarer than moderate fluctuations.
Ablation studies further highlighted the importance of the classification step. Removing it caused the strategy's return to drop to 42%—underperforming even a basic buy-and-hold approach. This suggests that identifying the type of news is a prerequisite for accurate financial reasoning.
Performance Evaluation
The CAR framework demonstrated clear superiority over both traditional benchmarks and simpler AI prompting methods. In back-testing, the CAR strategy achieved an 80.35% return with a Sharpe Ratio of 1.071.
As shown in the performance comparison, the CAR strategy was particularly effective at tracking abrupt gold price increases and avoiding the significant drawdowns that plagued simpler "Classify-only" or sentiment-based methods.
| Strategy | Return | Sharpe Ratio |
|---|---|---|
| Buy-and-Hold | 63.53% | 0.811 |
| One-Step (Simple Sentiment) | 63.44% | 0.902 |
| Classify (Two-Step) | 73.41% | 1.019 |
| Classify+Rethink (CAR) | 80.35% | 1.071 |
Conclusion and Future Work
The research demonstrates that structured prompting can effectively mitigate behavioral biases in LLMs, turning them into more reliable tools for financial decision-making. However, several challenges remain. The "alpha" or excess returns generated by such strategies may diminish as LLM-based trading becomes more widespread. Additionally, the method requires a high "reasoning floor," as smaller open-source models often struggle with the complex numerical scoring required.
Future research will explore more sophisticated portfolio management techniques, dynamic thresholding, and the application of this framework across different asset classes and languages.
Summary updated