Abstract
Financial professionals face significant challenges in processing the vast volume of daily news. This paper presents a four-stage Chain-of-Thought (CoT) framework designed to transform raw financial data into personalized, actionable investment insights. By employing structured reasoning and few-shot learning, the proposed system significantly reduces hallucinations and improves relevance. Experimental results demonstrate that this approach outperforms GPT-4o by 267% in BLEU scores and 90% in ROUGE-L scores, providing a robust tool for investor decision support.
1. Introduction
In the financial sector, information overload often obscures critical market signals. While Large Language Models (LLMs) offer potential for automated summarization, they frequently suffer from three primary limitations: factual hallucinations, a lack of domain-specific grounding (e.g., prioritizing narrative over quantitative metrics), and the generation of generic outputs that fail to account for individual investor preferences.
To address these issues, we propose a personalized CoT pipeline. This framework treats summarization as a multi-step reasoning task rather than a simple text-to-text translation, ensuring that summaries are both factually grounded and tailored to specific sector interests.
2. Methodology
The proposed framework utilizes a Mistral-7B-Instruct-v0.2 base model and operates through a four-stage sequential pipeline that moves from raw data to personalized reports.
2.1 Data Processing
The initial stage involves extracting text from financial PDFs. A character-ratio filter is applied to isolate English-language content, which is essential for processing bilingual reports common in global finance.
2.2 Initial and Enhanced Summarization
The second stage generates a structured narrative focusing on core entities, dates, and locations. This is followed by an "Enhanced Summarization" stage using few-shot learning. By providing the model with expert-curated examples, the system learns to prioritize quantitative developments—such as funding rounds and valuations—over general storytelling.
2.3 Personalized Response Generation
The final stage employs Binary Relevance Retrieval to filter articles based on user-defined keywords (e.g., "Renewable Energy"). The system then synthesizes relevant information into a final report containing key insights and specific investment recommendations.
3. Experimental Evaluation
The system was evaluated using a human-curated ground truth dataset developed by a financial analyst with over a decade of experience. Performance was measured against standard GPT-4o summaries using BLEU and ROUGE-L metrics.
3.1 Performance Results
The CoT framework demonstrated substantial quantitative improvements over the baseline, particularly in linguistic accuracy and relevance.
Key findings include:
- BLEU Score: A 267% improvement over GPT-4o (0.1786 vs. 0.0487).
- ROUGE-L Score: A 90% improvement (0.4028 vs. 0.2123).
- Retrieval Accuracy: The Binary Relevance method achieved 87.5% accuracy in identifying relevant articles, significantly higher than the 62.5% achieved by standard ranking methods.
Qualitative feedback from certified analysts confirmed that the CoT-generated summaries were more focused on critical quantitative data and exhibited fewer hallucinations compared to generic LLM outputs.
4. Conclusion and Future Work
The research highlights that treating summarization as a reasoning task significantly enhances the utility of LLMs in specialized domains like finance. While the current four-stage pipeline is computationally intensive and lacks real-time data integration, it provides a superior foundation for investor decision support. Future research will focus on optimizing the pipeline's efficiency and integrating live market data feeds to provide more dynamic insights.
Summary updated