Text-based inference of moral sentiment change

Jing Yi Xie, Renato Ferreira Pinto Junior, Graeme Hirst, Yang XuEMNLP 2019 · DOI

TL;DR

This paper introduces a novel text-based framework to track how societal moral sentiment towards concepts evolves over time, using diachronic word embeddings. By analyzing historical text corpora, the authors demonstrate the ability to infer shifts in moral relevance, polarity, and fine-grained moral dimensions, offering a scalable way to study societal moral evolution.

What Problem Does This Paper Address?

Understanding how moral sentiments change within a society is a fundamental question in social sciences, history, and philosophy. Traditionally, this has been studied through qualitative analysis or surveys, which are often limited in scale and scope. While NLP has made strides in detecting moral sentiment in current text, there's a significant gap in formally modeling and analyzing the diachronic aspect – how these sentiments evolve over long historical periods. This paper aims to bridge this gap by leveraging NLP techniques to automatically infer and analyze historical shifts in moral sentiment at scale, with minimal human intervention.

Methods

The core of the framework is built upon diachronic word embeddings. These are word embeddings that have been aligned across different time periods, allowing for the tracking of semantic changes. The authors propose a three-tier modeling framework:

  1. Moral Relevance: Distinguishing between concepts that are morally relevant and those that are not.
  2. Moral Polarity: Classifying relevant concepts as either positively or negatively perceived.
  3. Fine-grained Moral Dimensions: Categorizing concepts into specific moral foundations, drawing inspiration from Moral Foundations Theory (MFT), which includes categories like Care/Harm, Fairness/Cheating, etc.

They employ several probabilistic models (Centroid, Naïve Bayes, k-NN, KDE) to classify concepts within these tiers, using seed words from the Moral Foundations Dictionary (MFD) as labeled examples. The models exploit the semantic structure of the diachronic word embeddings to infer moral sentiment.

Datasets

  • Google N-grams: A large corpus of 8.5 x 10¹¹ tokens from English literature spanning 1800-1999.
  • COHA (Corpus of Historical American English): A smaller corpus of 4.1 x 10⁸ tokens, representative of American English from 1810-2009.
  • Moral Foundations Dictionary (MFD): A psycholinguistic resource providing seed words for MFT categories.
  • Warriner et al. (2013) valence ratings: A dataset of emotional valence ratings for approximately 14,000 English words, used to correlate with moral polarity predictions.
  • Pew Research Center's 2013 Global Attitudes survey: Data from 40 countries judging topics on acceptability, unacceptability, or as "not a moral issue," used for evaluating predictions of moral relevance and acceptability.

Key Insights and Ideas

  • Moral Sentiment is Embedded in Language: The fundamental premise is that societal moral views are reflected in the language used, and this can be captured by analyzing word embeddings over time.
  • Diachronic Embeddings are Key: Aligning word embeddings across historical periods is crucial for tracking semantic and moral shifts.
  • Abstract Concepts Change Morally Faster: The study found that abstract concepts tend to change their moral relevance at a higher rate than concrete concepts.
  • Nuanced Moral Trajectories: The framework can reveal complex historical trajectories, showing how concepts like "slavery" moved towards moral vice and "democracy" towards moral positivity over centuries.
  • Predictive Power: The model's predictions show significant correlations with human judgments of moral relevance and acceptability.

Key Results

  • High Classification Accuracy: Models achieved substantially better-than-chance accuracy in classifying moral relevance, polarity, and fine-grained moral categories across historical corpora. The Centroid model showed competitive accuracy with a simple, parameter-free design.
  • Correlations with Human Judgments: Significant Pearson correlations were found between model-predicted moral polarity and human valence ratings (Table 3).
  • Historical Trajectories: The framework successfully illustrated historical trends for concepts like "slavery" (moving towards moral vice around the 1860s) and "democracy" (moving towards moral positivity). The concept "gay" showed a shift in moral relevance around the 1930s, coinciding with its semantic change.
  • Concreteness Predicts Change: Multiple linear regression revealed that concreteness is a significant negative predictor of change in moral relevance, meaning abstract concepts change more rapidly in moral perception (Table 6).

Limitations and Future Work

  • Cultural and Linguistic Generalization: The current work focuses on English-speaking cultures. Future research should explore the framework's applicability to diverse cultures and linguistic backgrounds.
  • Interaction with Lexical Change: Investigating how moral sentiment change interacts with broader linguistic meaning change and lexical coinage is a promising direction.
  • Ground Truth: The authors acknowledge that direct "ground truth" for historical moral sentiment is not available, relying on indirect evidence and correlations.
  • Limited Topics for Human Judgment Comparison: The comparison with human judgments was limited by the number of available topics.

The authors suggest that their framework opens opportunities for applying NLP to characterize moral sentiment change in society, paving the way for more nuanced socio-scientific research.

Summary updated

BibTeX
@inproceedings{xie2019text,
  author = {Jing Yi Xie and Renato Ferreira Pinto Junior and Graeme Hirst and Yang Xu},
  title = {{Text-based inference of moral sentiment change}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2019},
  doi = {10.18653/v1/d19-1472},
}
Figure