TL;DR
This paper introduces SEAR, the first multimodal dataset designed to study social engineering (SE) attacks powered by augmented reality (AR) and large language models (LLMs). The dataset captures realistic interactions, revealing the alarming effectiveness of these new attack vectors in manipulating user trust and eliciting compliance, highlighting an urgent need for robust defenses.
What Problem Does This Paper Address?
The rapid advancement of AR and LLMs has opened new avenues for human-computer interaction, but it also presents a growing threat: sophisticated social engineering (SE) attacks. Unlike traditional SE methods that rely on static tactics, AR-LLM systems can dynamically adapt their behavior in real-time, analyzing environmental cues and user responses to craft highly personalized and manipulative interactions. This makes them significantly harder to detect and defend against. Existing research and datasets have largely focused on unimodal SE (e.g., text-based phishing) or isolated AR privacy risks, lacking a comprehensive resource that captures the multimodal dynamics of AR-LLM-driven SE. This gap hinders the development of effective detection and mitigation strategies.
Methods
The SEAR framework, used to generate the dataset, integrates three key components:
- AR Glass: Captures synchronized visual and audio data from the user's perspective, including facial expressions, vocal tones, and environmental context.
- Multimodal LLM (Gemma 3-12B): Processes the captured AR data and retrieves personal information (e.g., social media profiles) to build detailed, target-specific profiles.
- Social Agent: Employs an adaptive attack strategy, leveraging the LLM-generated profiles and real-time cues to engage in iterative response-refinement loops, aiming to build rapport and achieve SE objectives.
The dataset itself is structured around five empirically grounded components:
- Conversation Scene: Synchronized AR video/audio recordings of interactions.
- Personal Information: Collected social media and website data of SE targets.
- Social Profile: Multimodal LLM-generated profiles of the targets.
- Interactive Conversation: Full recordings of attacker prompts and SE target responses.
- Post-Interaction Survey: Subjective assessments from SE targets on trust, susceptibility, and overall experience.
Datasets
The paper introduces the SEAR Dataset as its primary contribution. This novel multimodal dataset comprises 180 annotated conversations from 60 participants in simulated adversarial scenarios. It captures synchronized AR visual/audio cues, environmental context, curated social media profiles, and subjective metrics. The dataset is designed to facilitate research in detecting AR-driven SE attacks, developing defensive frameworks, and understanding multimodal adversarial manipulation.
Key Insights and Ideas
- The Alarming Efficacy of AR-LLM SE: The combination of AR and LLMs significantly amplifies the effectiveness of social engineering attacks, making them more personalized and harder to resist.
- Multimodal Cues are Crucial: Beyond text, facial expressions, vocal tones, and environmental context play a vital role in building rapport and manipulating trust in SE scenarios.
- Dynamic Adaptation is Key: Unlike static phishing, AR-LLM systems can adapt their strategies in real-time, making them more agile and convincing.
- Trust Hijacking is Real: Participants showed a dramatic increase in trust after interacting with the SEAR system, indicating a significant vulnerability.
- Ethical Considerations are Paramount: The authors emphasize rigorous ethical safeguards, including anonymization and IRB compliance, for responsible dataset use.
Key Results
- High Compliance Rates: Participants exhibited alarmingly high compliance rates, with 93.3% clicking phishing links and 85% accepting calls.
- Significant Trust Increase: Post-interaction, 76.7% of participants reported an increase in trust towards the attacker.
- Effectiveness of SEAR Framework: The full SEAR framework (AR + LLM + Social Agent) significantly outperformed baseline conditions (bare conversation, AR + LLM only), with 76.7% rating it "Very Good" compared to lower scores for the baselines.
- Subjective Experience: Participants reported high satisfaction with SEAR across various dimensions, including relevance, naturalness, and AR comfort.
Limitations and Future Work
Limitations: * The study was conducted in controlled environments simulating real-world scenarios, which may not fully capture the complexities of natural interactions. * While the dataset is multimodal, the depth of analysis for each modality (e.g., specific nuances of vocal analysis) could be further explored.
Future Work: * Extend the coverage of SE strategies within the dataset. * Develop and test potential defense mechanisms against AR-LLM-driven SE attacks. * Encourage community adoption to accelerate research and build safer AR-LLM ecosystems.
Summary updated