Saying no but meaning yes: negation and sentiment analysis in Basque

Jon Alkorta, Koldo Gojenola, Mikel IruskietaEMNLP 2018

TL;DR

This paper investigates how negation affects sentiment analysis in Basque, a language where negation can strengthen, weaken, or even have no effect on a word's sentiment. The authors developed a set of linguistic rules using Constraint Grammar to identify these nuanced effects, achieving promising results in their evaluation.

What Problem Does This Paper Address?

Sentiment analysis aims to determine the emotional tone of text. A crucial aspect of this is understanding how words and phrases contribute to the overall sentiment. However, negation, a common linguistic phenomenon, can significantly alter the intended meaning and, consequently, the sentiment. For instance, saying "not good" is clearly negative, but what about "not bad"? This phrase, while containing a negator, often implies a neutral or even slightly positive sentiment. This paper tackles the challenge of accurately capturing these complex interactions between negation and sentiment, specifically within the Basque language, where such phenomena might be under-researched. Existing sentiment analysis tools often struggle with these subtleties, leading to inaccurate sentiment predictions.

Methods

The authors employed a Constraint Grammar (CG3) approach to formalize linguistic rules for identifying negation's effects on sentiment. This method involves defining a set of grammatical constraints to analyze text. They first built a corpus of Basque texts and identified negation instances. Then, they used a pre-existing semantic lexicon (Sentitegi) to assign initial sentiment orientation (SO) values to words. The core of their methodology involved:

  1. Corpus Creation: Extracting 359 negation instances from 96 Basque reviews across various topics.
  2. Polarity Extraction: Using a POS tagger and the Sentitegi lexicon to assign SO values (-5 to +5) to words.
  3. Linguistic Analysis: Manually analyzing how negation markers interact with words and phrases to strengthen, weaken, or have no effect on sentiment.
  4. Rule Development: Designing CG3 rules to detect these identified negation phenomena.
  5. Evaluation: Testing the developed rules on a separate set of 48 reviews using precision, recall, and F1-score.

Datasets

  • Basque Opinion Corpus: A corpus of 96 reviews used for initial analysis and rule development. This corpus covers six topics: movies, music, literature, politics, sports, and forecast.
  • Test Set: A separate set of 48 texts from the same Basque Opinion Corpus used for evaluating the developed CG3 rules.

The paper does not introduce a new benchmark dataset but rather utilizes and annotates an existing one for the specific task of negation analysis in Basque sentiment.

Key Insights and Ideas

  • Negation is Not Monolithic: The most significant insight is that negation in Basque doesn't simply reverse sentiment. It can actively strengthen positive sentiment (e.g., "not more beautiful" implying extreme beauty), weaken it (the most common effect), or have no effect at all.
  • Context is King: The impact of negation is highly dependent on its scope and the surrounding linguistic context. For example, the negator "ezin" (can not) can strengthen sentiment when modifying adjectives/adverbs but weaken it when negating verbs/nouns.
  • "Shift Negation" as a Model: The authors lean towards a "shift negation" approach, where negators have their own SO value (assigned as ±4 in their study) that interacts with the negated word's SO, rather than a simple polarity reversal ("switch negation"). This allows for more nuanced outcomes.
  • Lexicalized Structures and Conjunctions: Certain linguistic structures, like contrastive negations or conjunctions involving negation, can render the negation marker inert in terms of sentiment change.

Key Results

  • Overall Performance: The CG3 rules achieved an F1-score of 0.86 for identifying negation-related elements, with a precision of 0.93 and recall of 0.80.
  • Effect Breakdown:
    • Strengthening: Achieved a perfect 1.00 F1-score.
  • Weakening: Achieved an F1-score of 0.86.
  • No Effect: Achieved an F1-score of 0.98.
  • Frequency of Effects: Weakening was the most common effect, observed in 89.98% of cases. Strengthening occurred in 1.96% of cases, and no effect in 8.08%.

Limitations and Future Work

  • Limitations:
    • The authors acknowledge that the rules for weakening and scope analysis are less precise, with lower F1 scores. This is attributed to the unpredictability of scope components and length, and the challenge of handling punctuation-based constraints.
  • The current CG3 rules might need more syntactic information to accurately detect certain structures.
  • Future Work:
    • Implement the developed negation rules into an automatic Basque sentiment analysis tool.
  • Conduct further analysis of negation scope, potentially using Rhetorical Structure Theory (RST) to investigate if the position of a negator within a rhetorical structure impacts sentiment.
  • Refine rules for negation weakening and scope detection.

Summary updated

BibTeX
@inproceedings{alkorta2018saying,
  author = {Jon Alkorta and Koldo Gojenola and Mikel Iruskieta},
  title = {{Saying no but meaning yes: negation and sentiment analysis in Basque}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2018},
}
Figure