Minimally Supervised Learning of Affective Events Using Discourse Relations

Jun Saito, Yugo Murawaki, Sadao KurohashiEMNLP 2019 · DOI

TL;DR

This paper introduces a novel method for learning the sentiment polarity of "affective events" (events that impact people positively or negatively) by leveraging discourse relations between events. By propagating sentiment from a small seed lexicon through causal and concessive relationships in text, the model achieves strong performance with minimal supervision, even improving results when limited labeled data is available.

What Problem Does This Paper Address?

Recognizing affective events and their associated sentiment (positive or negative) is crucial for various Natural Language Processing (NLP) applications like dialogue systems and question answering. However, this is a challenging task because the sentiment of an event isn't always predictable from the words used to describe it. This "non-compositionality" means that understanding affective events often requires extensive world knowledge, which is difficult to acquire from small, manually labeled datasets. The paper aims to address this by developing a method that can learn affective event polarity with minimal human supervision, relying instead on the structure of discourse in raw text.

Methods

The core of the proposed method is to exploit discourse relations to propagate sentiment polarity. The authors focus on two specific relations: CAUSE and CONCESSION.

  • Discourse Relation Propagation: The key idea is that if event X1 causes event X2, and X2 is known to be positive, then X1 is likely also positive. Conversely, if X1 causes X2 and X2 is negative, X1 is likely negative. For CONCESSION, the polarity is reversed. Even when the polarity of X2 isn't known, the method assumes a tendency for events in a CAUSE relation to share polarity, and events in a CONCESSION relation to have opposite polarities.
  • Polarity Function: A neural network model is used to learn a polarity function, $p(x)$, which outputs a sentiment score between -1 (negative) and 1 (positive). This function uses an encoder (either BiGRU or BERT) to represent the event and a linear layer with a tanh activation.
  • Loss Functions: Three types of event pairs are used to train the model:
  • AL (Automatically Labeled Pairs): These pairs have a known polarity for the latter event (from the seed lexicon) and a discourse relation. The former event's polarity is then inferred based on the relation (same for CAUSE, opposite for CONCESSION). This provides "reference scores" for training.
  • CA (CAUSE Pairs): The seed lexicon doesn't match either event, but the discourse relation is CAUSE. The loss function encourages the two events to have similar polarities.
  • CO (CONCESSION Pairs): Similar to CA pairs, but the discourse relation is CONCESSION. The loss function encourages the two events to have opposite polarities.
  • The overall loss is a sum of these individual loss functions, with hyperparameters balancing their contributions.

Datasets

  • Raw Corpus: A Japanese web corpus was used to extract event pairs tagged with discourse relations.
  • Seed Lexicon: A small lexicon of 15 positive and 15 negative words was constructed.
  • AL, CA, and CO Datasets: These were automatically extracted from the raw corpus. The paper details the number of pairs for each type, with AL having 1 million pairs, and CA and CO having significantly more (5 million each).
  • ACP Corpus: This existing dataset, originally for sentiment analysis, was adapted for evaluation. It was used for both supervised training (6,000 labeled events) and as a benchmark for the semi-supervised method.

Key Insights and Ideas

  • Discourse Relations as Sentiment Connectors: The paper successfully demonstrates that discourse relations, particularly CAUSE and CONCESSION, can act as powerful implicit signals for sentiment propagation between events. This moves beyond simple word-level or phrase-level sentiment analysis.
  • Minimal Supervision is Effective: The method shows that a very small seed lexicon, combined with a large raw corpus and discourse structure, is sufficient to learn affective event polarities effectively. This significantly reduces the reliance on expensive manual annotation.
  • Handling Non-Compositionality: The approach implicitly learns to handle non-compositional expressions and idioms (like "肩を落とす" - to feel disappointed) by observing how their sentiment propagates through discourse.
  • BERT vs. BiGRU: While both encoders performed well, BERT's performance slightly decreased when incorporating CA and CO data, suggesting it might be more sensitive to the noise present in these automatically extracted pairs compared to BiGRU.

Key Results

  • The proposed method achieved high accuracy on the ACP test set, with the BiGRU encoder trained on AL+CA+CO reaching 0.866 and BERT reaching 0.835.
  • When trained with a small subset of labeled data (6,000 ACP events), the semi-supervised method (ACP+AL+CA+CO) significantly outperformed supervised training on the same small dataset, demonstrating its effectiveness in low-resource scenarios.
  • The model successfully learned to handle negation and idiomatic expressions without explicit supervision, as evidenced by example predictions. For instance, it correctly identified "道に迷う" (get lost) as negative (-0.771) and "道に迷わない" (don't get lost) as positive (0.835).

Limitations and Future Work

  • Noise in Discourse Relations: The authors acknowledge that automatically extracted discourse relations can contain noise, which can affect performance. They suggest that adding linguistically-motivated filtering rules could improve results.
  • Hyperparameter Sensitivity: The performance of the CA pairs was found to be more sensitive to noise than CO pairs, indicated by the hyperparameter values.
  • Future Work: The paper suggests that incorporating more discourse relations beyond CAUSE and CONCESSION, and exploring more sophisticated filtering mechanisms for noisy data, would be valuable future research directions.

Summary updated

BibTeX
@inproceedings{saito2019minimally,
  author = {Jun Saito and Yugo Murawaki and Sadao Kurohashi},
  title = {{Minimally Supervised Learning of Affective Events Using Discourse Relations}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2019},
  doi = {10.18653/v1/d19-1581},
}
Figure