Abstract
Text sentiment analysis is a critical task in public opinion monitoring and service evaluation. While statistical learning methods have historically performed well, they rely heavily on manual feature engineering, which is labor-intensive and lacks mobility. Neural networks offer a solution by reducing this dependence. However, standard architectures have limitations: Recurrent Neural Networks (RNNs) capture context but suffer from word-order bias, while Convolutional Neural Networks (CNNs) excel at local feature extraction via pooling but struggle with long-distance contextual dependencies. This paper proposes a hybrid sentiment analysis method combining R-CNN and C-RNN architectures through a learnable fusion gate. By integrating these structures, the model alleviates individual shortcomings and achieves superior performance across diverse datasets.
Introduction
Sentiment analysis in the modern network environment often struggles with the "bias vs. context" trade-off. RNNs are effective at capturing long-range context but tend to prioritize information at the end of a sequence, potentially ignoring key sentiment markers at the beginning. Conversely, CNNs excel at extracting local "n-gram" features (e.g., "not good") regardless of position but fail to understand the global relationship between distant phrases. To address these issues, we propose an ensemble model that captures both local patterns and global dependencies without manual intervention.
Methods
The core innovation is the "Ensemble with Fusion Gate" architecture, which integrates two distinct sub-networks to capture diverse semantic information.
R-CNN Sub-network
The R-CNN (Recurrent-Convolutional) layer utilizes a Bidirectional RNN to generate context-rich vectors for every word, capturing both preceding and succeeding information. These representations are then processed through a max-pooling layer to identify the most salient features across the entire document.
C-RNN Sub-network
The C-RNN (Convolutional-Recurrent) sub-network reverses the standard logic. It first employs a CNN to extract local features through sliding windows. Instead of immediate pooling, these features are treated as a new sequence and fed into a Long Short-Term Memory (LSTM) network. This allows the model to understand how local phrases relate to one another over long distances.
The Fusion Gate
Rather than simple averaging, the model employs a learnable gating mechanism. Similar to the gates in an LSTM, the Fusion Gate calculates a control ratio ($g$) based on input features, dynamically deciding the weight assigned to R-CNN versus C-RNN outputs for any given text.
Experiments and Results
The model was evaluated on three datasets: Ctrip Hotel (Chinese), the Stanford Sentiment Treebank (English), and a multi-domain Dianping dataset (Chinese) covering six fields.
Performance Metrics
The proposed ensemble consistently outperformed baselines such as SVM, standard RNNs, and the Recursive Neural Tensor Network (RNTN):
- Stanford Dataset: Achieved 46.1% accuracy on the 5-level sentiment task, surpassing the RNTN (45.7%).
- Dianping Dataset: Reached 47.6% on the 5-level task, a significant improvement over the RNTN's 42.6%.
- Negation Handling: In specialized "Negated Positive" tests (e.g., "not delicious"), the model achieved 81.7% precision, nearly doubling the performance of standard RNNs (43.3%).
Key Insights
- Architecture Diversity: R-CNN enhances individual words with context, while C-RNN extracts local patterns and models their global sequence. Combining them provides a more holistic semantic understanding.
- Dynamic Weighting: The fusion gate allows the model to be adaptive. If sentiment is driven by a specific local phrase, the gate favors C-RNN; if sentiment is buried in narrative structure, it favors R-CNN.
- Robustness: The model showed high stability in multi-domain tests where data for specific categories was sparse, demonstrating lower performance degradation than single-architecture models.
Conclusion
By combining R-CNN and C-RNN through a gating unit, this method effectively captures both local and global text features. While the model is computationally heavier than a simple CNN, its ability to handle complex linguistic structures and negations makes it a robust tool for sentiment analysis. Future work will focus on refining the gating mechanism and addressing multi-domain data sparsity.
Summary updated