TL;DR
This paper tackles the challenge of automatically identifying "face-saving" language, specifically in the context of political debates. By analyzing parliamentary question-and-answer pairs, the authors develop and evaluate computational models that can distinguish between genuine reputation defense and other types of discourse, demonstrating that contextual information significantly improves accuracy.
What Problem Does This Paper Address?
In social interactions, individuals and organizations often face criticism or accusations that threaten their "face" or reputation. Responding to these threats effectively, known as "face-saving" or "reputation defense," is crucial. While previous research has explored the strategies used in reputation defense, there's a gap in automatically analyzing the language employed. This paper aims to bridge that gap by developing computational models to identify reputation defense language within parliamentary debates, a setting rich with naturally occurring reputation threats and defenses. The authors specifically focus on Canadian parliamentary proceedings, where the oral question period serves as a structured environment for holding the government accountable.
Methods
The core of this research involves building and evaluating machine learning models for binary classification (reputation defense vs. non-defense).
- Feature Engineering: The researchers explored various linguistic features, including:
- LIWC (Linguistic Inquiry and Word Count): Analyzing word categories like anger, negative/positive emotions, achievement, and cognitive processes.
- NRC Word-Emotion Association Lexicon: Extracting scores for basic emotions (anger, fear, joy, etc.) and sentiments.
- NRC VAD Lexicon: Incorporating valence, arousal, and dominance scores.
- Vagueness Cue Words: Identifying words that might be used to soften or avoid direct confrontation.
- Bigrams: Pairs of consecutive words.
- Word Pairs: Analyzing interactions between words from both the question and answer.
- Models: A range of models were employed:
- Baselines: Majority class, SVM with unigram features (answers only, or questions+answers).
- Neural Networks: Convolutional Neural Networks (CNNs) and Long Short-Term Memory (LSTM) networks were used to process sequences of words.
- SVM with Word Pairs: A specific SVM model was trained on pairs of words from the question and answer to capture their interaction.
The key innovation lies in using the context of the exchange – specifically, the question posed – to help classify the answer as reputation defense.
Datasets
The primary dataset used is derived from the Lipad (Linked Parliamentary Data) dataset of Canadian parliamentary proceedings from 1994 to 2014.
- Initial Corpus: A balanced corpus of 9,048 question-and-answer pairs was created, with an equal number of opposition questions (considered reputation threats) and government backbencher questions (considered friendly non-threats).
- Government-Specific Datasets: For analyzing ideological differences, datasets were created for parliaments with Liberal governments (36th, 37th, 38th) and Conservative governments (39th, 40th, 41st). These datasets were not balanced in terms of question types.
- Balanced Cross-Parliament Datasets: For cross-parliament evaluation, balanced datasets of 3,400 question-answer pairs (1,700 of each class) were used.
Key Insights and Ideas
- Context is King: The most significant finding is that incorporating the question (the "reputation threat") into the analysis of the answer (the "reputation defense") dramatically improves classification accuracy. Models that consider both question and answer consistently outperform those that only look at the answer.
- Ideology Matters (but not that much): While there are differences in how Liberal and Conservative governments frame their defenses, models trained on one party's parliament and tested on another's still achieve high accuracy, suggesting the core language of defense is somewhat universal across political ideologies.
- Anger as a Signal: Among the emotion features, "anger" was found to be a significant contributor to identifying reputation defense, aligning with theories that suggest attacking the accuser can be a face-saving tactic.
- Vagueness and Hedging: Vagueness cue words and bigrams also proved useful features, supporting theories that suggest hedging language can be employed to avoid face-threatening acts.
Key Results
- High Accuracy: The best-performing models, particularly those using neural networks (CNNs and LSTMs) that consider both questions and answers, achieved accuracies and F1 scores above 98% on parliaments with Conservative governments and above 95% on Liberal governments in within-parliament evaluations.
- Cross-Parliament Performance: Even when models were trained on one party's parliament and tested on another's, the SVM model trained on word pairs (capturing question-answer interactions) achieved the best accuracy, reaching above 92%.
- Neural Networks Outperform Baselines: CNN and LSTM models generally outperformed simpler SVM and GRU models, especially when considering both question and answer context.
- Error Analysis: Most classification errors occurred when classifying answers to non-threat questions, suggesting that distinguishing between genuine defense and simple information provision or image enhancement can be challenging.
Limitations and Future Work
- Focus on Parliament: The current models are trained and evaluated on parliamentary data. Their performance on other domains (e.g., corporate communication, interpersonal disputes) is not yet known.
- Distinguishing Nuances: The error analysis highlighted that distinguishing between reputation defense and other forms of positive self-presentation (like enhancing one's image without a direct threat) can be difficult, especially for models relying solely on the answer.
- Future Work: The authors suggest applying their models to analyze human behavior and examine the effectiveness of reputation defense in various social settings. Further exploration into more nuanced classification tasks and different domains would be valuable.
Summary updated