Building Chinese Affective Resources in Valence-Arousal Dimensions

Liang-Chih Yu, Lung-Hao Lee, Shuai Hao, Jin Wang, Yunchao He, Jun Hu, K. Robert Lai, Xuejie ZhangNAACL 2016

TL;DR

This paper introduces the first comprehensive Chinese valence-arousal (VA) lexicon and corpus, crucial for fine-grained sentiment analysis beyond simple positive/negative classifications. The authors demonstrate that these new resources enable VA prediction comparable to existing English-language tools, paving the way for more nuanced cross-lingual sentiment analysis.

What Problem Does This Paper Address?

Traditional sentiment analysis often categorizes text as positive, negative, or neutral. However, human emotions are far more complex and can be better represented on continuous dimensions like valence (pleasantness/unpleasantness) and arousal (excitement/calm). While dimensional sentiment analysis offers richer insights, there's a significant lack of resources, particularly for non-English languages like Chinese. This paper addresses this gap by building and evaluating the first large-scale Chinese affective lexicon and corpus annotated with VA ratings.

Methods

The authors constructed two key resources:

  • Chinese Valence-Arousal Words (CVAW): This lexicon was built upon an existing Chinese affective lexicon (C-LIWC) and then annotated with VA ratings by five trained annotators. They used the Self-Assessment Manikin (SAM) model, which employs visual aids to help annotators assign scores on a 1-9 scale for both valence and arousal.
  • Chinese Valence-Arousal Text (CVAT): This corpus consists of 2,009 sentences extracted from various web text categories (news, forums, reviews). These sentences were annotated with VA ratings using a crowdsourcing platform, also employing the SAM model.

A crucial step in both constructions was a corpus cleanup procedure to remove outlier ratings and improper texts, ensuring higher data quality. For evaluation, they employed a simple aggregate-and-average method to predict sentence-level VA ratings using the CVAW lexicon. This involves averaging the VA ratings of matched words within a sentence.

Datasets

  • CVAW (Chinese Valence-Arousal Words): A newly created lexicon containing 1,653 Chinese words annotated with valence-arousal ratings.
  • CVAT (Chinese Valence-Arousal Text): A newly created corpus of 2,009 Chinese sentences annotated with crowd-sourced valence-arousal ratings.
  • ANEW (Affective Norms for English Words): Used for comparison in the evaluation section, this is an existing English-language lexicon.
  • English Forum Discussions: A dataset of 20 English forum discussions used for comparative evaluation.

Key Insights and Ideas

  • Arousal is Harder to Distinguish: The study found that annotators consistently had higher error rates (both MAE and RMSE) when rating arousal compared to valence. This suggests that distinguishing between different levels of excitement/calm is more challenging than pleasantness/unpleasantness.
  • Dimensionality Matters for Chinese: The creation of these VA resources is a significant step towards enabling more nuanced sentiment analysis for Chinese text, moving beyond simple polarity detection.
  • Cross-Lingual Comparability: The performance of the Chinese resources in predicting VA ratings was found to be comparable to existing English resources, indicating the generalizability of the VA dimensional approach.
  • Text Length and VA Prediction: The results suggest that for longer texts (like English forum discussions), prediction might be more robust, while shorter texts (like CVAT sentences) present a different challenge.

Key Results

  • The CVAW lexicon contains 1,653 words with mean valence of 4.49 (SD 1.81) and mean arousal of 5.48 (SD 1.26).
  • The CVAT corpus contains 2,009 sentences with mean valence of 4.83 (SD 1.37) and mean arousal of 5.05 (SD 0.95).
  • When predicting VA ratings of the CVAT corpus using CVAW words, the performance (MAE, RMSE, and correlation 'r') was comparable to using English resources (ANEW) on English forum discussions. For example, the MAE for valence prediction was 1.20 for both CVAW/CVAT and ANEW/Forum, and for arousal, it was 1.52 and 0.85 respectively.
  • The study found that for valence prediction, the Laptop review category in CVAT achieved the lowest error rate, while News and Hotel had higher correlations. For arousal prediction, Book reviews yielded the lowest error rate, with Hotel and Book also showing better correlations.

Limitations and Future Work

  • Arousal Difficulty: The paper acknowledges that distinguishing arousal is more difficult, which is reflected in higher error rates.
  • Limited English Comparison: While a comparison with English resources was made, it was on different types of texts (forum discussions vs. web sentences), which might influence the direct comparability.
  • Future Work: The authors plan to focus on building useful dimensional sentiment applications based on these newly constructed resources. This could include applications like more sophisticated product review analysis, political stance detection, or even emotion-aware dialogue systems.

Summary updated

BibTeX
@inproceedings{yu2016building,
  author = {Liang-Chih Yu and Lung-Hao Lee and Shuai Hao and Jin Wang and Yunchao He and Jun Hu and K. Robert Lai and Xuejie Zhang},
  title = {{Building Chinese Affective Resources in Valence-Arousal Dimensions}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2016},
}
Figure