Abstract
LLM Agents are becoming central to intelligent systems. However, their deployment raises serious safety concerns. Existing defenses largely rely on "Safety Checks", which struggle to capture the complex semantic risks posed by harmful user inputs or unsafe agent behaviors—creating a significant semantic gap between safety checks and real-world risks. To bridge this gap, we propose a novel defense framework, ALRPHFS (Adversarially Learned Risk Patterns with Hierarchical Fast & Slow Reasoning). ALRPHFS consists of two core components: (1) an offline adversarial self-learning loop to iteratively refine a generalizable and balanced library of risk patterns, substantially enhancing robustness without retraining the base LLM, and (2) an online hierarchical fast & slow reasoning engine that balances detection effectiveness with computational efficiency. Experimental results demonstrate that our approach achieves superior overall performance compared to existing baselines, achieving a best-in-class average accuracy of 80% and exhibiting strong generalizability across tasks.
Introduction
As LLM agents evolve from simple chatbots into autonomous systems capable of web browsing and code execution, their attack surface expands significantly. Current defense strategies primarily utilize Safety Checks, which typically fall into two categories: static handcrafted rules or dynamic runtime guards. However, these methods often suffer from a semantic gap—they fail to recognize harmful intent when actions appear benign in isolation but constitute a high-level risk pattern when viewed holistically.
ALRPHFS aims to close this gap by shifting the focus from forbidden keywords to high-level semantic risk patterns. By teaching the system to recognize abstract patterns of risk, it becomes more resilient to the wide variety of real-world attacks that bypass traditional rigid filters.
Methodology: The ALRPHFS Framework
The ALRPHFS framework operates through a dual-phase architecture designed for both robustness and efficiency.
1. Offline: Adversarial Pattern Learning
Instead of relying on manual rule-writing, ALRPHFS builds a Risk Pattern Database through an automated pipeline:
- Pattern Extraction & Deduplication: High-level abstractions are extracted from harmful trajectories and refined using DBSCAN clustering and Greedy Multi-Medoids Selection to ensure a compact, diverse library.
- Self-Learning Adversarial Optimization: A "Red Team" LLM generates attack variants to identify weaknesses in the current pattern library, while a "Blue Team" refines the patterns. This loop includes "easily misclassified benign samples" to improve discriminative balance and reduce false positives.
2. Online: Hierarchical Fast & Slow Reasoning
Inspired by dual-process theory, the online engine balances speed and depth:
- Fast Thinking: Employs hybrid retrieval (Vector similarity + BM25) to match queries against the database. High-confidence matches trigger immediate interception.
- Slow Thinking: For ambiguous cases, a multi-step reasoning chain performs deep, evidence-based risk estimation, considering historical context and potential downstream impacts.
Experimental Evaluation
The framework was evaluated across diverse benchmarks, including R-Judge for safety awareness, TrustAgent for unintended risks, and the Agent Security Bench (ASB) for intended attacks (e.g., prompt injection and memory poisoning).
Key Results
ALRPHFS demonstrated superior performance across all metrics:
- Detection Accuracy: The framework achieved a best-in-class average accuracy of 83.13% on intended attacks (ASB), significantly outperforming baselines like Llama-Guard 3 and AGrail.
- Robustness: On the TrustAgent dataset, it reduced the Attack Success Rate (ASR) to 21.79%, a marked improvement over standard model-based defenses.
- Efficiency: Despite the added reasoning layers, ALRPHFS maintained a competitive token footprint, consuming fewer tokens than other self-learning defenses like AGrail while providing higher safety margins.
Discussion and Conclusion
ALRPHFS successfully bridges the semantic gap in agent defense by utilizing abstracted risk patterns rather than specific strings. This abstraction makes the system harder to bypass via simple paraphrasing. While the "Slow Thinking" mode introduces some latency, the hierarchical approach ensures that deep reasoning is only applied when necessary, optimizing the safety-per-dollar ratio. Future work will explore streamlining the reasoning process and moving toward real-time database updates during live deployment.
Summary updated