Helpful or Hierarchical? Predicting the Communicative Strategies of Chat Participants, and their Impact on Success

Farzana Rashid, Tommaso Fornaciari, Dirk Hovy, Eduardo Blanco, Fernando Vega-RedondoEMNLP 2020 · DOI

Introduction

While Natural Language Processing (NLP) has advanced significantly in sentiment analysis and topic modeling, capturing the "strategic" nuances of human interaction remains a challenge. Communication is more than an exchange of information; it involves building reputations, asserting power, and offering support. This study addresses the gap in understanding how communicative styles—such as equality versus hierarchy—manifest in professional peer networks, specifically among entrepreneurs. The researchers investigate whether these subtle interaction styles can be reliably automated and if they correlate with external professional achievements.

Methods and Datasets

The study utilizes a unique dataset of approximately 140,000 chat interactions from an online business course involving 5,000 entrepreneurs across 49 African countries. A subset of 5,500 interactions was manually annotated across four dimensions:

  • Cooperativeness: Friendly/cooperative vs. competitive.
  • Advice: Providing suggestions or guidance.
  • Motivational: Encouraging others.
  • Equality: Treating others as peers vs. asserting a hierarchy.

To analyze this data, the researchers employed a two-step computational approach. First, they used a Support Vector Machine (SVM) to classify communicative dimensions based on linguistic features like TF-IDF and sentiment lexica. Second, they developed a Multitask Learning (MTL) Convolutional Neural Network (CNN) to predict real-world success indicators (e.g., business ownership) by leveraging communicative styles as auxiliary tasks.

Key Insights: The Dynamics of Professional Communication

Analysis revealed significant patterns in how entrepreneurs interact. A notable "Business-Cooperative" trade-off was observed: interactions focused on specific business details tended to be less cooperative and more transactional or competitive, evidenced by a correlation of -0.42. Conversely, "Motivational" styles showed strong positive correlations with "Cooperative" (0.61), "Advice" (0.56), and "Equal" (0.54) styles, suggesting that encouraging behavior often coincides with helpfulness and peer-level engagement.

Correlation matrix showing relationships between business indicators (B), success (S), and communicative styles (C, M, E, A).

Results and Performance

The manual annotation process yielded reliable ground truth data, with an average raw agreement of 83% across categories.

Inter-rater agreement metrics (Raw, Kappa, and Alpha) for the four communicative dimensions.

The automated SVM model performed exceptionally well at identifying these styles, significantly outperforming majority baselines. For instance, it achieved F-measures of 0.89 for "Motivational" and 0.86 for "Advice." In terms of predicting business success, the MTL CNN proved effective, achieving an F-measure of 0.65 for predicting business proposal submissions. These results confirm that communicative strategy is a strong signal for real-world professional outcomes.

Limitations and Future Directions

The study acknowledges limitations such as data imbalance, particularly in rare categories like "competitive" interactions. Additionally, the findings are specific to the demographic of African entrepreneurs in a structured course. Future research could explore more complex pragmatic features and longitudinal graph-based analyses to understand how these interaction patterns evolve over time within professional networks.

Summary updated

BibTeX
@inproceedings{rashid2020helpful,
  author = {Farzana Rashid and Tommaso Fornaciari and Dirk Hovy and Eduardo Blanco and Fernando Vega-Redondo},
  title = {{Helpful or Hierarchical? Predicting the Communicative Strategies of Chat Participants, and their Impact on Success}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2020},
  doi = {10.18653/v1/2020.findings-emnlp.214},
}
Figure