A Simple and Effective biLSTM Approach to Aspect-Based Sentiment Analysis in Social Media Customer Feedback

Simon ClematideKONVENS 2018 · OpenReview

Abstract Aspect-Based Sentiment Analysis (ABSA) in social media presents unique challenges due to linguistic noise, data imbalance, and the need for joint prediction of aspects and sentiments. This paper presents a refined bidirectional Long Short-Term Memory (biLSTM) sequence labeling approach that achieves state-of-the-art performance on the GermEval 2017 German social media dataset. By utilizing hybrid character-token embeddings and a novel "dummy token" trick to unify document-level and token-level tasks, the proposed system outperforms more complex architectures. Results demonstrate significant improvements in Opinion Target Expression (OTE) detection and competitive performance in joint aspect-sentiment classification.

Introduction

Sentiment analysis has evolved from simple document-level polarity classification to the more granular task of Aspect-Based Sentiment Analysis (ABSA). ABSA aims to identify specific aspects of a service or product (e.g., "punctuality" in a railway context) and the sentiment expressed toward those aspects. In the domain of social media, this task is complicated by informal language, typos, and highly imbalanced datasets.

This paper, originally presented at KONVENS 2018, argues that "over-engineering" complex architectures with external lexicons is often unnecessary. Instead, a well-tuned, relatively simple neural architecture can capture the necessary linguistic patterns implicitly, provided it is robust to the noise inherent in social media data.

Methodology

The core of the proposed system is a bidirectional Long Short-Term Memory (biLSTM) network. Several key design choices contribute to its effectiveness:

Hybrid Embeddings

To address the "long tail" of rare words and typos common in social media, the model employs a dynamic embedding strategy. Frequent words are mapped to standard token-level embeddings. For rare words (those falling below a frequency threshold), the system switches to a character-level biLSTM representation. This allows the model to derive meaning from sub-word components, making it robust to misspellings like "zugig" (drafty) vs "zug" (train).

The "Dummy Token" Trick

ABSA typically involves both document-level classification (identifying the overall aspect) and token-level extraction (identifying the specific words referring to that aspect). The author unified these tasks by appending a "dummy token" to the end of every input sequence. This token is assigned the document-level aspect label, allowing the entire problem to be framed as a single sequence labeling task.

Simplified Labeling and Ensembling

The system uses a simplified IO (Inside, Outside) labeling scheme rather than the standard IOB scheme. Given the sparsity of aspect labels in the dataset, the "Beginning" marker was found to be redundant. To further improve performance, the final system is an ensemble of 24 models. A voting threshold of 33% was used to favor recall, ensuring that the system identifies more aspects in the heavily imbalanced dataset.

Experimental Setup

The system was evaluated using the GermEval 2017 ABSA dataset, which contains German customer feedback regarding "Deutsche Bahn" (the national railway). The dataset is characterized by a high degree of imbalance, with 68.5% of labels belonging to the "GENERAL" category and a dominance of neutral sentiments. Evaluation was performed on two test sets: a "synchronic" (SYN) set from the same time period as the training data, and a "diachronic" (DIA) set from a later period to test temporal generalization.

Results and Discussion

The system was evaluated on two primary tasks: Task C (Joint Aspect/Sentiment) and Task D (Opinion Target Expression).

Task C: Joint Aspect and Sentiment

In Task C, the goal is to predict the aspect and sentiment simultaneously. The system's performance, particularly the "Our A:S" variant using fine-grained labels, surpassed previous shared task participants and official baselines.

Table 1: Performance on Task C (Joint Aspect and Sentiment).

As shown in Figure 1, while the LT-ABSA system achieved higher scores for aspect-only (A) detection, the proposed system ("Our A:S") achieved the best results for the combined aspect and sentiment (A:S) metric on both the synchronic and diachronic test sets.

Task D: Opinion Target Expression (OTE)

The most significant gains were observed in Task D, which involves identifying the exact spans of text that express an opinion.

Table 2: Performance on Task D (Opinion Target Expression).

Figure 2 highlights the substantial improvement over the state-of-the-art. The proposed model improved F-scores by 2.7 to 14.3 percentage points compared to previous systems like LT-ABSA and Mishra. The performance was consistent across both "exact" and "overlap" matching criteria, demonstrating the model's precision in span detection.

Conclusion

The success of this biLSTM-based approach underscores the value of simplicity and robust handling of linguistic noise in social media analysis. By framing document-level and token-level tasks as a unified sequence labeling problem and utilizing character-level embeddings for rare words, the system achieved state-of-the-art results without relying on complex external features. Future work could improve performance further through more sophisticated tokenization and better handling of multi-label documents.

Summary updated

BibTeX
@inproceedings{clematide2018simple,
  author = {Simon Clematide},
  title = {{A Simple and Effective biLSTM Approach to Aspect-Based Sentiment Analysis in Social Media Customer Feedback}},
  booktitle = {KONVENS},
  year = {2018},
  url = {https://openreview.net/forum?id=VW0tIAQtcP},
}
Figure