Abstract
In this chapter, we present additional application scenarios in the financial field using the methods discussed, with a particular focus on tasks related to generative AI.
Introduction
While Generative AI has demonstrated remarkable proficiency in general-purpose conversation, the financial sector demands specialized capabilities beyond simple sentiment analysis. Current financial Natural Language Processing (NLP) faces three primary challenges:
- Temporal Blindness: Difficulty in predicting the duration of impact for events, particularly long-term Environmental, Social, and Governance (ESG) commitments.
- Noise in Opinion Ranking: The challenge of identifying high-quality, professional insights amidst the volume of social media and analyst reports.
- The "Math Gap": The inherent struggle of Large Language Models (LLMs) with quantitative reasoning and multi-hop data querying.
Methodology
To address these gaps, we propose two architectural enhancements:
- Professionalism-Aware Pre-finetuning: A middle-tier training step that teaches models to distinguish between amateur and professional discourse, embedding domain-specific logic into the model weights.
- Retrieval-Augmented Generation (RAG) for ESG: A scheme that combines GPT-4o with a retrieval mechanism to provide historical context for temporal predictions, using cosine similarity to select relevant training instances.
ESG Impact Duration and RAG Performance
Analysis of ESG data reveals that over 50% of ESG-related news (excluding Korean data) pertains to events with impacts lasting longer than five years. This "long game" nature of ESG necessitates models that can reason over extended temporal horizons. By implementing a RAG-enhanced GPT-4o model, we observed significant improvements in F1 scores for impact duration prediction across multiple languages.
Professionalism and Profitability
Our research suggests that "professionalism" in language serves as a strong proxy for investment profitability. Models pre-finetuned to recognize professional linguistic patterns significantly outperform standard models in identifying profitable suggestions on social media. Specifically, professionalism-aware pre-finetuning reached a 23.39% Mean Profitability (MPP) when ranking social media posts, surpassing standard financial models.
Numeracy and Creative Agents
Beyond sentiment and professionalism, we addressed "number sense" through specific pre-finetuning tasks, such as numerical magnitude comparison. This approach led to a 26.40% improvement in quantitative prediction for financial comments using RoBERTa.
Furthermore, we explored AI's role in generating economic indices (e.g., EPU Index). While humans excel in country-specific context and explainability, AI-generated indices demonstrate considerable predictability for variables like non-farm employment, offering a scalable solution for global economic research where human experts may be unavailable.
Conclusion and Future Work
Despite these advancements, significant bottlenecks remain. Models continue to struggle with multi-hop reasoning across complex database tables (DBQR task). Future research should focus on human-agent collaboration and the development of multi-scale models capable of decomposing complex financial workflows into manageable sub-tasks.
Summary updated