TalkDown: A Corpus for Condescension Detection in Context

Zijian Wang, Christopher Potts

Introduction

Condescension is a subtle, "caustic" form of communication that relies more on social context and power dynamics than overt negativity. Unlike hate speech or profanity, condescension is often "polite" on the surface—it can even involve insincere praise. Its essence lies in pragmatic presuppositions: the speaker assumes a higher social status than the listener, while the listener disagrees.

Because of this, an utterance that is friendly between peers might be highly condescending when said by a customer to a clerk. Most existing sentiment analysis tools fail here because they look at sentences in isolation. This paper introduces TALKDOWN, a new dataset of nearly 5,000 human-labeled Reddit interactions, and demonstrates that incorporating discourse context into BERT-based models significantly improves the detection of these complex linguistic acts.

Example of a condescending interaction on Reddit, illustrating the distinction between the preceding context and the specific quoted span that triggered the accusation.

The TALKDOWN Corpus

The primary contribution of this work is the TALKDOWN Corpus, derived from Reddit data spanning 2006–2018. To build this dataset, researchers scraped 2.62 million "Comment/Reply" pairs, specifically targeting replies where a user quoted a portion of the previous comment and followed it with a condescension-related accusation (e.g., "Stop being so condescending").

The final corpus consists of 4,992 valid human-labeled instances, with 65.2% labeled as positive for condescension. Each instance includes the original comment, the specific quoted span, and the reply containing the accusation. To ensure label quality, the authors used Amazon Mechanical Turk and an Expectation-Maximization (EM) approach to aggregate labels while accounting for individual worker reliability.

Modeling and the Importance of Context

The researchers fine-tuned BERT (both Base and Large versions) to detect condescension. A key experimental variable was the input provided to the model: some versions only saw the "Quoted" text, while others saw the "Quoted" text plus the "Context" (the text appearing before the quote in the original comment).

Results showed that adding the preceding linguistic context provided a 3–4% boost in macro-F1 score. This confirms that condescension is often invisible without knowing what prompted the remark. However, the quoted span itself remains a strong signal, suggesting that when people accuse others of condescension, they are usually able to pinpoint the exact phrase that "crossed the line."

Performance comparison of BERT models using different input combinations, highlighting the significant gain from including discourse context.

Community Norms and Insights

The researchers ran their model across different subreddits and found that condescension rates vary significantly based on community rules. For example, r/AskWomen has lower rates than r/AskMen, likely because r/AskWomen is strictly curated to promote respectful discussion.

Interestingly, the subreddit r/funny showed the highest rates of predicted condescension. The authors suspect the model might be confusing irony and sarcasm with condescension, highlighting a remaining challenge in nuance detection. Sarcasm often involves "punching up," whereas condescension is defined by "talking down."

Conclusion

The BERT-Large model using both Quoted text and Context achieved a Macro-F1 of 0.684 on imbalanced data. The models remained effective even when tested on highly imbalanced datasets (1:20 ratio), which is critical for deploying such systems in live online communities. Future work may incorporate "user identities" to provide the social context that text alone lacks, such as a user's history or their relationship with the person they are replying to.

Summary updated

Abstract

Condescending language use is caustic; it can bring dialogues to an end and bifurcate communities. Thus, systems for condescension detection could have a large positive impact. A challenge here is that condescension is often impossible to detect from isolated utterances, as it depends on the discourse and social context. To address this, we present TalkDown, a new labeled dataset of condescending linguistic acts in context. We show that extending a language-only model with representations of the discourse improves performance, and we motivate techniques for dealing with the low rates of condescension overall. We also use our model to estimate condescension rates in various online communities and relate these differences to differing community norms.
BibTeX
@inproceedings{wang2019talkdown,
  author = {Zijian Wang and Christopher Potts},
  title = {{TalkDown: A Corpus for Condescension Detection in Context}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2019},
  doi = {10.18653/v1/d19-1385},
}
Figure