Abstract
Sentiment analysis remains a critical challenge in Natural Language Processing (NLP) due to the inherent complexities of human language, such as sarcasm and irony. While the prevailing trend in the field focuses on scaling Transformer-based models, this research explores an alternative path: Combinatorial Fusion Analysis (CFA). By intelligently merging a fine-tuned RoBERTa transformer with traditional machine learning models, researchers achieved a new state-of-the-art accuracy of 97.072% on the IMDb benchmark, demonstrating that cognitive diversity between models can be more effective than raw scale.
Introduction
The "bias-variance dilemma" often plagues sentiment classification; complex models may overfit specific patterns, while simpler models may fail to capture nuanced linguistic features. Traditional ensemble methods, such as simple averaging, often overlook the unique "perspectives" that heterogeneous models bring to a dataset. This study addresses this gap by applying CFA to exploit the unique strengths of diverse architectures, moving beyond simple performance-weighted combinations to focus on the informational diversity of the ensemble members.
Methodology
The research team constructed an ensemble consisting of four distinct base models to ensure high feature-level diversity:
- RoBERTa (Model A): A state-of-the-art transformer utilizing deep subword embeddings.
- SVM (Model B): A classical linear classifier.
- XGBoost (Model C): A gradient boosting framework.
- Random Forest (Model D): A traditional ensemble of 25,000 decision trees.
Combinatorial Fusion Analysis (CFA)
The core innovation lies in the use of CFA to quantify and leverage "Cognitive Diversity" (CD). The process involves mapping each model's output using a Rank-Score Characteristic (RSC) Function, which visualizes how a model assigns confidence scores relative to its internal rankings.
By measuring the difference in how two models rank the same data, the researchers calculated the CD metric. This allowed for the implementation of Weighted Combination by Diversity Strength (WCDS), where models are weighted based on the unique information they contribute rather than their individual accuracy alone.
Key Results and Discussion
The application of diversity-weighted fusion yielded significant performance gains over both individual models and standard ensemble techniques. The ensemble achieved a record-breaking accuracy of 97.072%, surpassing the previous state-of-the-art for the IMDb dataset.
Insights on Diversity
- Complementary Biases: Analysis revealed that RoBERTa exhibited a slight bias toward positive predictions (higher recall), while the SVM was more conservative (higher precision). The CFA fusion moderated these biases, resulting in a more balanced and robust classifier.
- Efficiency: Once base models are trained, the CFA fusion step is computationally negligible, requiring only simple arithmetic. This provides a scalable path to high-performance NLP without the necessity for massive GPU clusters.
- Diversity vs. Performance: WCDS consistently outperformed performance-weighted fusion (WCP). This confirms that a collection of models that make different types of errors is more valuable than a collection of the highest-performing models that fail in the same way.
Conclusion
This research demonstrates that the intelligent fusion of heterogeneous models through Combinatorial Fusion Analysis can significantly outperform individual state-of-the-art transformers. By prioritizing cognitive diversity, the proposed method achieved a new benchmark on the IMDb dataset while maintaining computational efficiency. Future work will explore the application of this framework to more diverse datasets, such as social media and financial news, and investigate the inclusion of newer architectures like XLNet to further push the boundaries of cognitive diversity in sentiment classification.
Summary updated