Introduction
Traditional financial theories often struggle to account for the extreme volatility observed in energy markets. While supply and demand are fundamental drivers, human emotion—ranging from investor panic to consumer optimism—plays a significant role, particularly during "black swan" events like the COVID-19 pandemic. Analyzing this sentiment is challenging because social media and online forums are flooded with messy, unstructured data that varies wildly in style and quality.
Standard sentiment analysis tools frequently fail to capture the nuances of Chinese-language comments or the long-term dependencies in text that signal a user's true market outlook. To address this, researchers have developed a hybrid deep learning model, BERT-BiLSTM, designed to provide robust technical support for market forecasting by effectively "reading the room" of the energy market.
Methodology
The proposed model utilizes a "combination forecasting method" that stacks two powerful architectures to handle the complexities of natural language.
BERT for Contextual Embeddings
The upstream component of the model is BERT (Bidirectional Encoder Representations from Transformer). Unlike traditional models like word2vec that assign a fixed vector to a word regardless of context, BERT generates dynamic representations based on surrounding words. The researchers employed a simplified Chinese training model with 12 layers and 110 million parameters to extract high-quality word vectors that capture the semantic essence of investor comments.
BiLSTM for Sequence Modeling
The downstream component is BiLSTM (Bidirectional Long Short Term Memory). While BERT excels at understanding context, BiLSTM is specialized in processing sequences. By analyzing text both forward and backward, it captures long-term dependencies and the grammatical logic of human language better than standard Recurrent Neural Networks (RNNs).
In this hybrid approach, the BERT output is fed directly into the BiLSTM layer, followed by a Softmax layer for final sentiment classification. This synergy allows the model to cover both the "micro" statistical characteristics of nearby words and the "macro" logic of the entire sentence.
Data and Experimental Setup
The study utilized a dataset of approximately 100,000 comments from investors and consumers published on the Chinese Internet between late 2019 and the first half of 2020.
- Preprocessing: Data was cleaned to remove invalid entries such as empty text or punctuation-only strings.
- Balancing: To address a natural imbalance (a 1:5 ratio of positive to negative/neutral comments), the researchers used upsampling and downsampling to achieve a 1:2 ratio for the training set.
- Split: The dataset was divided into training, validation, and test sets using a 7:1:2 ratio.
Results and Discussion
The hybrid BERT-BiLSTM model demonstrated superior performance across all key metrics when compared to standalone BERT and LSTM models.
| Model | Accuracy | Recall (Positive) | F1 Score |
|---|---|---|---|
| BERT-BiLSTM | 0.8620 | 0.7078 | 0.6357 |
| BERT | 0.8559 | 0.5576 | 0.5684 |
| BiLSTM | 0.7775 | 0.0747 | 0.1025 |
A critical finding was the "Recall Gap." While standalone BERT achieved decent accuracy, its recall for positive sentiment (Label 1) was only 0.5576. The addition of the BiLSTM layer boosted this to 0.7078, indicating that sequence modeling is essential for identifying specific sentiment orientations in complex financial discourse. Conversely, the standalone BiLSTM failed significantly (recall of 0.0747), proving that sequence models require high-quality feature extraction from BERT to handle raw social media text.
Conclusion
The BERT-BiLSTM model offers a more accurate way to monitor sentiment in the energy market, providing valuable insights for investors and policymakers. Future work will focus on scaling the model with larger datasets and leveraging more powerful computing clusters to refine predictive accuracy for real-time decision-making.
Summary updated