TL;DR
This paper proposes a novel, language-agnostic approach to sentiment analysis for social media by translating foreign language tweets into English and then leveraging pre-trained English word embeddings. This method achieves competitive accuracy on Slavic languages without requiring language-specific lexicons or extensive training data.
What Problem Does This Paper Address?
Sentiment analysis, the task of determining the emotional tone of text, is crucial for understanding public opinion in areas like marketing, politics, and social science. While significant progress has been made, most existing methods and datasets are heavily skewed towards English. This creates a barrier for analyzing sentiment in other languages, especially those with fewer linguistic resources. Developing language-specific sentiment analysis tools is time-consuming and resource-intensive, often requiring the creation of specialized lexicons or large, annotated datasets. This paper tackles the challenge of performing sentiment analysis on multilingual corpora, specifically focusing on Slavic languages, by proposing a method that bypasses the need for language-specific resources.
Methods
The core of the proposed method is a two-step process:
- Translation to English: Documents in their original Slavic languages (Polish, Slovenian, Croatian) are translated into English using Google's Neural Machine Translation (GNMT) system. This leverages GNMT's ability to produce contextually relevant English translations.
- Vectorial Representation and Aggregation: The translated English texts are then processed to create a single vector representation for each document. This is achieved by:
- Using a pre-trained Word2Vec model (trained on Google News) to obtain a dense vector for each word in the translated document.
- Aggregating these word vectors into a single document vector by computing the elementwise sum of all word vectors and then dividing by the number of words in the document. This approach avoids the computational cost of padding variable-length sequences common in other deep learning methods.
Finally, standard machine learning classifiers (Gradient Boosting Trees, Random Forest, Support Vector Machines, and Deep Neural Networks) are trained on these aggregated document vectors to perform the sentiment classification.
Datasets
The study utilizes existing tweet corpora for three Slavic languages:
- Polish: 2794 tweets (1397 positive, 1397 negative)
- Slovenian: 4272 tweets (2312 positive, 1950 negative)
- Croatian: 3554 tweets (2129 positive, 1425 negative)
These datasets were sourced from the European research infrastructure CLARIN. The corpora for Slovenian and Croatian are noted as imbalanced. The authors preprocessed the tweets by removing URLs, emojis, digits, and punctuation marks.
Key Insights and Ideas
- Leveraging Translation as a Bridge: The paper demonstrates that machine translation can effectively bridge the gap between languages for sentiment analysis, allowing the reuse of powerful English-language NLP tools.
- Simplicity and Generality: The proposed method is remarkably simple and general. It doesn't require language-specific lexicons or training of language-specific word embeddings, making it adaptable to various languages with available translation services.
- Averaged Word Vectors as a Powerful Representation: Despite its simplicity, aggregating word vectors from a pre-trained model provides a sufficiently rich representation for sentiment classification, outperforming some more complex methods.
- Competitive Performance without Language-Specific Resources: The approach achieves respectable accuracy levels, comparable to or exceeding some state-of-the-art methods that rely on language-specific resources, highlighting the efficacy of the translation-based strategy.
Key Results
The study reports classification accuracies for the tested machine learning methods on the translated tweet corpora. The best classifiers achieved overall accuracies exceeding 76% for Polish, Slovenian, and Croatian. Specifically:
- Polish: The best accuracy was achieved by Gradient Boosting Trees (78.46%).
- Slovenian: Gradient Boosting Trees also performed best (76.10%).
- Croatian: Random Forest and Gradient Boosting Trees achieved the highest accuracies (86.32% and 86.26%, respectively).
The paper also presents ROC curves, indicating that tree-based methods (Random Forest and Gradient Boosting Trees) generally outperformed Support Vector Machines and Deep Neural Networks across the datasets. The recall measure was lower for the negative class in imbalanced corpora (Croatian and Slovenian), which the authors attribute to data imbalance.
Limitations and Future Work
The authors acknowledge several limitations and suggest future directions:
- Translation Quality: While Google's Neural Machine Translation is effective, its accuracy can still be a limiting factor, especially for nuanced language or specific domains.
- Data Imbalance: The imbalanced nature of the Slovenian and Croatian datasets impacted recall for the negative class.
- Discrepancies in Accuracy: The variation in accuracy across languages suggests that translation quality and labeling imperfections might play a role.
- Future Work: The authors plan to test their method on other Slavic languages and are working on creating their own pre-labeled, balanced dataset to further improve the approach and investigate these discrepancies. They also note that their setup does not involve unsupervised learning.
Summary updated