TL;DR
This paper introduces a novel deep learning architecture that uses Convolutional Neural Networks (CNNs) to analyze individual emails and Long Short-Term Memory networks (LSTMs) to capture the temporal order of messages, significantly improving the prediction of power relations in organizational communication. Their model achieves state-of-the-art accuracy on predicting power dynamics within email threads and across entire email exchanges.
What Problem Does This Paper Address?
Understanding social dynamics and relationships between people is a crucial aspect of human interaction. In professional settings, power structures are particularly important, influencing communication patterns and potentially leading to issues like incivility. While Natural Language Processing (NLP) has shown promise in analyzing language to infer these power relations, existing methods often struggle with capturing the nuances of how power is expressed in emails, especially considering the sequential nature of conversations. This paper aims to bridge this gap by developing a more sophisticated NLP model that can accurately predict the direction of power between individuals based on their email exchanges.
Methods
The authors propose three novel neural network architectures to tackle this problem:
- Batched-CNN: This approach treats all emails from one person to another as a single unit, feeding them into a CNN. This is analogous to prior SVM-based methods that aggregate features.
- Separated-CNN: Here, each individual email is processed by its own CNN, allowing the model to capture local patterns within each message before merging them. This approach separates the analysis of messages from sender A to receiver B, and vice-versa.
- Sequential-CNN-LSTM: This is the most advanced model, designed to account for the temporal order of emails. It uses CNNs to process individual emails and then feeds the sequence of CNN outputs into an LSTM for each participant. This allows the model to learn from the chronological flow of the conversation.
All models also incorporate non-lexical features, such as the average number of recipients and words per email, which are known to be indicative of power dynamics.
Datasets
The study utilizes the Enron Email corpus, specifically a version that includes organizational power relations for 13,724 employee pairs and reconstructed email thread structures. The authors mask greetings and signatures to avoid bias. They consider two problem formulations:
- Per-Thread: Predicting power within a single email thread between two interacting participants.
- Grouped: Predicting power based on all emails exchanged between two individuals across the entire corpus.
The dataset is split into training, development, and testing sets for both formulations.
Key Insights and Ideas
- Temporal Order Matters: The Sequential-CNN-LSTM model's superior performance on the Per-Thread task highlights the importance of considering the chronological sequence of emails in understanding power dynamics. Conversations evolve, and the order in which information is exchanged can reveal power shifts.
- Individual Email Nuances are Crucial: The Separated-CNN model's strong performance on the Grouped task suggests that analyzing individual emails for subtle linguistic cues of power is highly effective, even when the temporal structure is less pronounced.
- Beyond Surface-Level Features: The paper moves beyond simple word n-grams by using deep learning to capture more complex linguistic manifestations of power within emails.
- Challenges in Power Expression: The qualitative analysis reveals that power isn't always overtly expressed. Subordinates might display power through expertise, and superiors might use polite language, making accurate prediction challenging and highlighting the need for sophisticated models.
Key Results
- Per-Thread Task: The Sequential-CNN-LSTM model achieved an accuracy of 80.4%, a significant 10.1% improvement over the previous state-of-the-art (73.0%).
- Grouped Task: The Separated-CNN model achieved an accuracy of 83.0%, outperforming traditional methods by a substantial margin (70.0%).
- All proposed neural network models significantly outperformed the SVM baseline in both task formulations.
Limitations and Future Work
The authors acknowledge that their model sometimes struggles with subtle expressions of power, particularly when text features alone are insufficient. They suggest future work could involve:
- Granular Feature Learning: Training CNNs on individual sentences rather than entire emails to capture mid-level power indicators.
- Hybrid Architectures: Incorporating structural features (like gender, temporal position) at different levels of the architecture, merging them with per-individual LSTMs or per-email CNNs as appropriate.
- Cross-Corpora Validation: Testing their models on additional datasets, such as the Avocado Research Email Collection, to assess generalizability.
Summary updated