Stylistic variation over 200 years of court proceedings according to gender and social class

Stefania Degaetano-OrtliebNAACL 2018

TL;DR

This paper introduces a novel information-theoretic approach using relative entropy to analyze stylistic variation in language over 200 years of court proceedings, focusing on the interplay of gender and social class. The findings reveal distinct diachronic shifts in language use, particularly for higher-class women, suggesting a reflection of changing social roles.

What Problem Does This Paper Address?

Understanding how language varies based on social factors like gender and social class is a long-standing goal in sociolinguistics. While recent advances in Natural Language Processing (NLP) have enabled predicting social context from language, much of this work focuses on contemporary data (e.g., social media) and often overlooks the diachronic dimension and the influence of social class. This paper addresses this gap by investigating how language use has evolved over two centuries in court proceedings, specifically examining the nuanced interplay between gender and social class. The authors hypothesize that as women's social positions shifted in the mid-19th century, this might be reflected in their linguistic choices.

Methods

The core of the paper's methodology lies in the application of relative entropy, also known as Kullback-Leibler Divergence. This information-theoretic measure quantifies the difference between two probability distributions. In this context, it's used to compare the language use of different social groups (defined by gender and class) across various time periods.

The authors use relative entropy to compare probability distributions at two linguistic levels:

  • Lexical level: This involves comparing the distribution of individual words.
  • Grammatical level: This is approximated by comparing the distribution of part-of-speech (POS) trigrams.

A key innovation is that they use entire linguistic levels (lexis and grammar) rather than pre-selected features, allowing for a more comprehensive and data-driven discovery of stylistic variation. The relative entropy values indicate how "apart" two groups are linguistically, with higher values signifying greater divergence. The asymmetry of relative entropy is leveraged to understand the directionality of these differences. Additionally, the contribution of individual linguistic units (words or POS trigrams) to the overall divergence is analyzed, providing insights into which specific elements are most distinctive for each group.

Datasets

The primary dataset used in this study is the Old Bailey Corpus (OBC). This corpus consists of transcribed court proceedings from London, spanning from 1720 to 1913. It is a manually socio-linguistically annotated dataset, providing information on gender, social class (using the HISCLASS standard), speaker role, and temporal periods. For their analysis, the authors filtered the corpus to include only victim and witness roles to control for speaker bias, as there were no female judges or lawyers in the dataset. The corpus is part-of-speech tagged with CLAWS 7.

Key Insights and Ideas

  • Diachronic shifts reflect social change: The study provides evidence that changes in women's social positions, particularly from the mid-19th century onwards, are mirrored in their language use, leading to increased divergence from other groups.
  • Relative entropy as a powerful tool: The paper demonstrates the effectiveness of relative entropy for capturing subtle linguistic differences across social variables and over long time spans, offering a more nuanced view than traditional methods.
  • Lexical and grammatical divergence: Stylistic variation is observed at both lexical and grammatical levels, with distinct patterns emerging for different social groups.
  • Higher-class women's evolving style: Female higher-class speakers, in particular, show a shift towards a more nominal and informational style over time, diverging from the more verbal and involved style of female lower-class speakers.
  • Asymmetry reveals communicative effort: The asymmetric nature of relative entropy highlights how the effort required to understand one group's language by another can differ, suggesting that male higher-class speakers might have found it easier to understand female higher-class speakers than vice versa.

Key Results

  • Lexical Level: From 1700 to 1800, female higher class (FH) showed less divergence from lower classes (FL, ML) than from male higher class (MH). After 1800, FH became less distinct from MH but more distinct from lower classes.
  • Grammatical Level: POS trigram analysis revealed that female higher class (FH) diverged more from female lower class (FL) over time. Their language use became more similar to male groups (MH, ML) than to FL. Specifically, FH developed a more nominal style, while FL maintained an involved verbal style.
  • Word Clouds: Visualizations of word contributions showed that FH used more business-oriented vocabulary and words related to marriage, while FL used vocabulary related to the household and objects. Stylistically, FH adopted a nominal style with distinctive use of nouns and determiners, while FL used more personal pronouns and verbs.

Limitations and Future Work

The authors acknowledge several limitations and suggest avenues for future research:

  • Focus on specific roles: The analysis was limited to victim and witness roles due to the absence of female judges or lawyers in the dataset. Future work could explore other roles if available.
  • Broad social class categories: The study used broad "higher" and "lower" social class distinctions. The OBC is annotated on a scale of 1-13, allowing for more granular distinctions in future analyses.
  • Fixed time periods: The comparison was done across fixed fifty-year periods. Future work could investigate specific time spans where changes are more pronounced.
  • Broader group analysis: While the focus was on female higher class, the authors are currently analyzing the development of all groups and plan to extend this to other roles within the trials.
  • More fine-grained distinctions: The authors suggest that the OBC's detailed social class annotations could be used for more nuanced comparisons.

Summary updated

BibTeX
@inproceedings{degaetanoortlieb2018stylistic,
  author = {Stefania Degaetano-Ortlieb},
  title = {{Stylistic variation over 200 years of court proceedings according to gender and social class}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2018},
}
Figure