TL;DR
This paper explores combining ELECTRA, a cost-efficient encoder, with GPT-4o, a powerful LLM, for sentiment analysis. They found that augmenting GPT-4o's prompts with ELECTRA's predictions significantly boosted performance for non-fine-tuned GPT models, offering a cost-effective way to improve accuracy. However, fine-tuning GPT models diminished this collaborative benefit.
What Problem Does This Paper Address?
Sentiment analysis, the task of understanding opinions in text, has seen rapid advancements with both bidirectional transformers like BERT and ELECTRA, and large language models (LLMs) like GPT. While fine-tuned transformers excel at specific tasks, LLMs offer impressive zero-shot and few-shot capabilities. The challenge lies in effectively combining these complementary strengths. Specifically, this paper investigates how to leverage the nuanced sentiment understanding of fine-tuned encoders (ELECTRA) to enhance the performance of LLMs (GPT-4o), particularly in a cost-effective manner, and whether this collaboration is beneficial when the LLM is also fine-tuned.
Methods
The researchers experimented with various ways to integrate ELECTRA's output into GPT-4o's input. This included providing ELECTRA's predicted class label, class probabilities, and even similar review examples as part of the prompt. They utilized the Declarative Self-improving Python (DSPy) framework to manage prompt engineering and experimentation. Four models were used: ELECTRA Base, ELECTRA Large, GPT-4o, and GPT-4o-mini. Experiments involved both zero-shot GPT, fine-tuned GPT, and collaborative scenarios where ELECTRA's predictions were fed to GPT.
Datasets
The study used a merged dataset combining movie reviews from the Stanford Sentiment Treebank (SST-3) and business reviews from DynaSent Rounds 1 and 2. The SST dataset was adapted to a three-way classification (positive, neutral, negative).
Key Insights and Ideas
- Collaboration Pays Off (for non-fine-tuned LLMs): Augmenting GPT-4o's prompts with ELECTRA's predictions significantly boosted performance, especially when GPT-4o was not fine-tuned. This suggests a powerful synergy where ELECTRA acts as a knowledgeable assistant.
- Fine-tuning GPT Dampens Collaboration: When GPT models were fine-tuned, the benefit of adding ELECTRA's predictions disappeared, and in some cases, performance even decreased. This implies that fine-tuning might make GPT less receptive to external guidance.
- Simplicity is Key (for prompt augmentation): Simply providing the predicted label from ELECTRA was often as effective, or more effective, than including probabilities or few-shot examples when augmenting GPT prompts.
- Fine-tuned GPTs Shine: When GPT models were fine-tuned, they achieved the highest overall performance, with GPT-4o-mini FT being a particularly strong contender at a much lower cost than the full GPT-4o.
- Cost-Effectiveness Matters: The paper highlights cost-effective strategies, such as using ELECTRA Base FT with GPT-4o-mini (not fine-tuned) for the best cost/performance ratio.
Key Results
- Augmenting GPT-4o-mini (not fine-tuned) with ELECTRA Base FT predictions improved macro F1 by +3.22 points, achieving the lowest cost/performance ratio ($0.12/F1 point).
- Fine-tuned GPT-4o-mini FT achieved a macro F1 of 86.70, while GPT-4o FT-M reached 86.99, at a significantly lower cost than other high-performing models.
- Fine-tuned ELECTRA Large outperformed base GPT models, offering a strong local option.
- When GPT models were fine-tuned, including ELECTRA predictions actually decreased performance. For example, GPT-40-mini FT's score dropped from 86.70 to 81.06 when augmented with ELECTRA Base FT predictions.
Limitations and Future Work
- Cost Evaluation: The cost analysis focused on fine-tuning costs, not ongoing inference costs for API-based models.
- Model Scope: The study was limited to OpenAI's GPT-4o models; open-source LLMs were not evaluated for fine-tuning costs.
- Prompt Optimization: While DSPy was used, further optimization of prompts could be explored.
- Understanding "Why": The paper acknowledges that further research is needed to understand why fine-tuned GPT models ignore ELECTRA's predictions.
- Generalization: Future work could explore different datasets, domains, and model pairings, as well as investigate fine-tuning GPTs on ELECTRA's output representations.
Summary updated