ADVERT: An Adaptive and Data-Driven Attention Enhancement Mechanism for Phishing Prevention

Linan Huang, Shumeng Jia, Emily Balcetis, Quanyan ZhuIEEE Trans. IFS 2022 · DOI

TL;DR

This paper introduces ADVERT, a novel human-centric system that uses real-time eye-tracking data to generate adaptive visual aids, guiding users' attention to prevent them from falling for phishing attacks. By modeling human attention as a dynamic feedback control system and employing reinforcement learning, ADVERT significantly improves phishing recognition accuracy.

What Problem Does This Paper Address?

Cybersecurity is constantly challenged by human vulnerabilities, particularly innate ones like bounded attention and rationality. Phishing attacks exploit these weaknesses, leading to significant data breaches and security compromises. While existing solutions focus on training or rule enforcement to address acquired vulnerabilities, they fall short against innate ones. This paper tackles the problem of "inattention" – a key innate vulnerability – by developing a system that actively assists users in their email vetting process, aiming to make them less susceptible to deceptive tactics.

Methods

ADVERT employs a multi-faceted approach:

  • Eye-Tracking and Visual State Modeling: Real-time eye-gaze data is collected and processed. This data is then abstracted into "Areas of Interest" (AoIs) within an email. A "Visual State Transition Model" is developed to characterize the user's gaze dynamics across these AoIs, representing their attention flow.
  • Attention Level Metrics: System-level metrics, specifically "Cumulative Attention Level" (CAL), are derived from the visual state transitions. These metrics are then quantized into "Attention States" (AS) to represent the user's attentiveness.
  • Adaptive Visual Aid Generation (Q-Learning): A Q-learning framework is used to learn an optimal policy for generating visual aids. These aids (e.g., highlighting, warnings) are dynamically adapted based on the user's current attention state, aiming to guide their focus and sustain engagement. This involves two feedback loops: one for short-term attention enhancement and another for long-term phishing prevention.
  • Meta-Adaptation (Bayesian Optimization): To fine-tune the system's hyperparameters (like visual aid generation period and attention thresholds), Bayesian Optimization is employed. This meta-level tuning efficiently searches for optimal configurations to maximize phishing recognition accuracy.

Datasets

The study utilizes a dataset collected from human subject experiments conducted at New York University [5]. This dataset includes:

  • 160 undergraduate students.
  • 12 different emails, some posing cyber threats and others legitimate.
  • Eye-tracking data (gaze locations and pupil diameters) recorded at 60Hz.
  • Participant ratings on the likelihood of taking solicited actions.

The paper also mentions generating synthetic data based on the experimental data to demonstrate the ADVERT framework.

Key Insights and Ideas

  • Human Attention as a Dynamic System: The paper frames human attention during email vetting as a stochastic feedback control system, allowing for quantitative modeling and manipulation.
  • Visual Aids as Cognitive Modifiers: ADVERT demonstrates that strategically applied visual aids can directly influence users' visual behaviors and, consequently, their cognitive states, leading to better security decisions.
  • Short-term vs. Long-term Adaptation: The system distinguishes between immediate attention enhancement (short-term feedback) and overall phishing prevention (long-term adaptation), using different feedback loops for each.
  • Meta-Adaptation for Robustness: Bayesian Optimization is crucial for finding optimal system parameters, making ADVERT robust to variations in user behavior and email content.
  • Main Content is Key: The study highlights that the "main content" AoI in an email is crucial for phishing detection, exhibiting the highest concentration score and lowest decay rate in attention.

Key Results

  • Improved Accuracy: Without visual aids, participants achieved an average phishing recognition accuracy of 74.6%. With ADVERT's visual aids, this accuracy increased to a minimum of 86%.
  • Meta-Adaptation Gains: Further tuning of hyperparameters through meta-adaptation improved accuracy from 86.8% to 93.7% in less than 50 tuning stages. The most significant improvements (88.7% to 91.4%) occurred within the first 3 tuning stages.
  • Efficiency and Robustness: The meta-adaptation process was shown to be efficient, with significant gains achieved quickly. The system also demonstrated robustness, with small variances in accuracy across tuning stages.
  • Hyperparameter Sensitivity: The study found that the attention threshold has a small impact on phishing recognition, while the period length for visual aid generation has a periodic impact.

Limitations and Future Work

  • Dataset Demographics: The initial dataset was limited to 160 undergraduate students. Future work suggests recruiting more diverse participants through crowdsourcing platforms.
  • Email Variety: The dataset contained only 12 unique emails, which may not cover all phishing scenarios. However, the authors argue this is sufficient for demonstrating the system-level control of attention.
  • Data Exclusion: Approximately one-third of the eye-tracking data was excluded due to low validity scores or participant inattentiveness. Future work could address this by improving eye-tracking devices, refining experimental settings, and recruiting more participants.
  • Sophisticated Visual Support: The authors propose designing more sophisticated visual support systems that can dynamically determine when and how to generate visual aids, moving beyond periodic generation.
  • VR/AR Integration: Embedding ADVERT into VR/AR technologies is suggested to mitigate human vulnerabilities in simulated deception scenarios.
  • Incentives and Pressure: Incorporating factors like time pressure and rewards for accurate phishing identification is proposed as a future direction.

Summary updated

BibTeX
@article{huang2022advert,
  author = {Linan Huang and Shumeng Jia and Emily Balcetis and Quanyan Zhu},
  title = {{ADVERT: An Adaptive and Data-Driven Attention Enhancement Mechanism for Phishing Prevention}},
  journal = {IEEE Trans. IFS},
  year = {2022},
  doi = {10.1109/tifs.2022.3189530},
}
Figure