Introduction
Social media has become a primary arena for political discourse, making the automated analysis of public opinion a critical task for researchers. However, standard sentiment analysis often falls short when dealing with polarized political topics. This paper addresses two specific challenges: Stance Detection and Gender Inference, within the context of the IBEREVAL 2017 competition.
Unlike sentiment analysis, which categorizes text as positive, negative, or neutral, stance detection determines whether a user is "for" or "against" a specific target—in this case, Catalan independence. A tweet may express negative sentiment toward a politician while maintaining a positive stance toward independence, making this a nuanced classification task. Additionally, since gender is rarely explicit on Twitter, the authors explore linguistic cues to infer user gender in a multilingual environment involving both Spanish and Catalan.
Methodology
The researchers from IIIT Hyderabad implemented a supervised classification pipeline centered on Support Vector Machines (SVM) with a Radial Basis Function (RBF) kernel. The system's strength lies in its multi-layered feature engineering approach:
- Character N-grams (n=1 to 3): These capture sub-word patterns, effectively handling typos, slang, and morphological variations typical of social media text.
- Word N-grams (n=1 to 5): These provide local context and capture specific political phrasing.
- Stance and Gender Indicative Tokens: A custom scoring formula was developed to identify tokens strongly correlated with specific labels. For instance, words appearing predominantly in "FAVOR" tweets were assigned higher weights, allowing the model to learn domain-specific jargon automatically without manual dictionaries.
To optimize performance, the team utilized Chi-squared ($\chi^2$) feature selection to retain only the most statistically significant markers. Pre-processing steps included hashtag decomposition (e.g., converting #CatalanIndependence into separate tokens), which significantly increased the semantic information available to the model.
Experimental Setup
The study utilized the official IBEREVAL 2017 dataset, comprising 4,319 training tweets and 1,081 test tweets in both Spanish and Catalan. The classification tasks involved three stance labels (FAVOR, AGAINST, NONE) and two gender labels (FEMALE, MALE).
Results and Discussion
The efficacy of the proposed features was evaluated using 10-fold cross-validation on the training set. The results demonstrated that combining all features—character n-grams, word n-grams, and indicative tokens—yielded the best performance across both languages and tasks.
As shown in Table 1, character n-grams were particularly effective for Spanish, achieving nearly the same accuracy as the full feature set for stance detection. In the Catalan subset, the "All features" model reached its peak performance during the training phase.
Despite strong cross-validation results (reaching 81.53% for Catalan stance as seen in Table 2), the system's performance on the unseen test set was more modest, with a macro-average F-score of 0.46 for stance detection in both languages. Gender detection proved particularly challenging in Catalan, dropping to 44.59% on the test set, which is below the baseline of random guessing. This discrepancy suggests that linguistic markers for gender in Catalan may be more subtle or that the training data lacked representativeness for the test distribution.
Conclusion
The LTRC IIITH system demonstrates the power of sub-word features and custom indicative tokens in analyzing polarized political discourse. While the approach was robust for stance detection, the difficulties encountered with Catalan gender inference highlight the need for more sophisticated linguistic modeling. Future work will explore Part-of-Speech (POS) tagging, word embeddings (e.g., Word2Vec), and deep learning architectures to better capture the complexities of short-form multilingual text.
Summary updated