Social Engineering Attack Detection in Corporate Communications

Hanzla Hamid, Pappu Jha, Nick RahimiarXiv 2026 · arXiv

TL;DR

This paper explores a shift in phishing detection from surface-level text analysis to the identification of psychological manipulation. By building interpretable, lexicon-based models grounded in Cialdini’s principles of persuasion, the authors demonstrate that while psychological features are less accurate than raw-text classifiers, they provide a vital "assistive signal" that explains how an attacker is attempting to manipulate a target.

What Problem Does This Paper Address?

Social engineering attacks—like phishing, pretexting, and baiting—are notoriously difficult to detect because they don't rely on technical vulnerabilities but on human psychology. Traditional security filters often fail because they look for specific keywords or technical signatures rather than the underlying intent of the message.

The authors argue that in a corporate environment, a "black box" classification (e.g., "this email is 98% likely to be malicious") isn't enough. Security analysts and employees need to understand the persuasive strategy being used—whether it's an appeal to greed, a manufactured sense of urgency, or an attempt to build false trust—to effectively triage threats and improve security awareness.

Methods

The research follows a two-pronged approach:

  1. Baseline Classification: The authors first established a high-performance baseline using standard NLP techniques: TF-IDF vectorization paired with Logistic Regression and Naive Bayes. These models achieved near-perfect accuracy by processing thousands of lexical dimensions.
  2. Psychological Lexicon Modeling: To prioritize interpretability, the authors developed a pipeline that compresses emails into roughly a dozen psychological dimensions. They hand-curated lexicons for categories like urgency, trust, authority, greed, curiosity, and social connection.
  3. Leakage Resistance: A key methodological contribution is the refinement of "Model 2." The authors identified that certain features (like "technical_discussion" or "sexual_pleasure") were artifacts of specific datasets rather than generalizable traits of phishing. By removing these "leakage-prone" categories, they created a more robust, standardized model.
  4. Statistical Validation: They utilized the Mann-Whitney U test and Cliff’s delta to ensure that the differences in feature distributions between malicious and benign emails were statistically significant and possessed meaningful effect sizes.

Datasets

The study utilized a massive, deduplicated collection of 82,078 emails aggregated from seven different Kaggle datasets, including:

  • Enron: Real-world corporate communications.
  • Ling & SpamAssassin: General spam and legitimate filters.
  • Nazario & Nigerian Fraud: Specific collections of known phishing and "419" scams.
  • CEAS_08: A standard benchmark for email filtering.

The authors emphasized deduplication (removing 408 identical emails) to prevent data leakage between training and testing folds.

Key Insights and Ideas

  • The "Why" vs. the "What": While raw-text models are more accurate, lexicon models are more useful for human-in-the-loop systems. A lexicon profile can tell an analyst: "This email is flagged because it combines high 'trust' building with a 'greed' incentive."
  • The Fear Paradox: Surprisingly, features like Fear and Authority were more prevalent in legitimate corporate emails. The authors attribute this to common workplace jargon (e.g., "risk," "problem," "department," "team") appearing more frequently in internal Enron emails than in the spam-heavy malicious datasets.
  • Trust as a Malicious Predictor: In this study, Trust was the strongest positive predictor for malicious intent. Attackers often spend significant effort building rapport or credibility before making a "request for action."
  • Urgency is Overrated: While "urgency" is a hallmark of social engineering theory, the data showed it also appears frequently in legitimate corporate reminders (deadlines, meeting invites), giving it a lower standalone predictive value than expected.

Key Results

The baseline TF-IDF models performed exceptionally well, with the Logistic Regression model reaching 98.53% accuracy.

However, the primary focus was the Leakage-Resistant Lexicon Model (Model 2). As shown in the results, this model achieved a ROC-AUC of 0.7397 on unseen data. While lower than the baseline, it relies on just 11 interpretable features rather than thousands of words.

TABLE IV: Model 2 (Leakage-Resistant Feature Set), Stan- dardized: Main Result

As seen in Figure 2, Trust (Odds Ratio 2.02) and Greed (Odds Ratio 1.59) were the most significant indicators of a malicious email, while Neutral Formality was a strong indicator of a legitimate one.

Statistical testing confirmed these trends. As shown in Figure 6, the mean frequency of "trust" and "greed" terms was significantly higher in malicious emails compared to legitimate ones.

TABLE X: Features Significantly Higher in Malicious Emails (Mann-Whitney U Test)

Limitations and Future Work

  • Dataset Aging: Many of the sources (like Enron) are decades old. Modern "Business Email Compromise" (BEC) attacks may use different linguistic patterns than the older spam/fraud samples used here.
  • English-Only: The study was limited to English text.
  • Missing Metadata: The models did not consider attachments, links, or sender metadata, which are critical components of modern phishing.
  • Future Directions: The authors suggest moving beyond email to platforms like Slack and Microsoft Teams, and exploring how Large Language Models (LLMs) might be used to identify these psychological triggers with even higher precision while maintaining explainability.

Summary updated

Abstract

Social Engineering attacks continue to pose a significant risk to the corporate world with a growing level of sophistication. It is difficult to distinguish between authentic and phishing communication via email, chat, and phone. They pose a risk of exposing trade secrets and confidential communications and can create a chain of attack from lower-level employees up to executives. Therefore, we study this growing threat using real-world datasets to build baseline machine-learning models. Our TF-IDF models classify emails as phishing or safe, achieving accuracy above 90%. We build on this by exploring whether psychological manipulation and persuasion indicators can be measured to play a significant role in detecting phishing. Instead of relying solely on surface-level lexical features, we derive interpretable lexicon-based indicators of urgency, trust, greed, and other persuasion cues from the combined email dataset. By evaluating these manipulation-oriented features on their own, we measure how much signal they carry (ROC-AUC of 0.74 on a held-out test split) and test whether they transfer to held-out email sources.
BibTeX
@misc{hamid2026social,
  author = {Hanzla Hamid and Pappu Jha and Nick Rahimi},
  title = {{Social Engineering Attack Detection in Corporate Communications}},
  year = {2026},
  eprint = {2611.14897},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.14897},
}
Figure