Explainable and Fine-Grained Safeguarding of LLM Multi-Agent Systems via Bi-Level Graph Anomaly Detection

Junjun Pan, Yixin Liu, Rui Miao, Kaize Ding, Yu Zheng, Quoc Viet Hung Nguyen, Alan Wee-Chung Liew, Shirui Pan

Abstract

As Large Language Model (LLM) Multi-Agent Systems (MAS) gain autonomy in complex domains like software engineering and social simulation, they become increasingly vulnerable to adversarial exploits. Traditional Graph Anomaly Detection (GAD) methods often fail to detect "camouflaged" attacks that hide malicious intent within seemingly benign dialogue. This paper introduces XG-Guard, an unsupervised framework that employs bi-level graph anomaly detection to identify malicious agents at both the sentence and token levels. By utilizing dynamic theme-based detection and correlation-guided fusion, XG-Guard provides high-precision security and human-readable explanations for flagged behaviors.

1. Introduction

The rise of LLM Multi-Agent Systems has introduced a new frontier for collaborative AI, but also a significant security risk. In these systems, agents communicate to solve tasks, share memory, and utilize external tools. However, a single compromised agent can propagate misinformation, "poison" shared memory, or execute unauthorized tool calls—such as exfiltrating sensitive data to an external address.

Current defense mechanisms primarily rely on Graph Anomaly Detection to identify outliers. However, these methods suffer from two critical limitations: coarseness and a lack of explainability. Most GAD tools operate at the sentence level, potentially missing subtle "trigger phrases" embedded in long responses. Furthermore, they often act as black boxes, providing binary labels without justifying why an agent was deemed malicious.

2. The XG-Guard Framework

XG-Guard addresses these gaps through a multi-faceted, unsupervised approach designed to catch subtle anomalies without requiring expensive labeled datasets.

2.1 Bi-Level Agent Encoding

To combat the issue of coarseness, XG-Guard processes agent communication through two parallel streams. The sentence-level stream captures the holistic semantic meaning of a response, while the token-level stream focuses on individual words. This dual-focus ensures that even a single malicious word or unauthorized API call hidden within a 500-word response can be isolated and detected.

2.2 Graph-Aware Learning and Topology

Because agents operate within a network, XG-Guard utilizes Graph Neural Networks (GNNs) to model communication topology (e.g., Chain, Tree, Star, or Random). To prevent "over-smoothing"—where an agent's unique behavior is averaged out by its neighbors—the system incorporates skip connections. This preserves the distinct behavioral signatures necessary for anomaly detection.

2.3 Dynamic Theme-Based Detection

Rather than comparing agents against a static baseline of "normal" behavior, XG-Guard calculates a dynamic Theme Prototype for every conversation. This recognizes that "normal" dialogue in a coding task differs significantly from a creative writing task. Agents whose responses deviate significantly from the current discussion's central theme are flagged as potential anomalies.

2.4 Correlation-Guided Fusion and Contrastive Learning

To minimize false positives, XG-Guard employs a covariance-based mechanism to align sentence and token scores. If a token appears anomalous but the sentence context is benign, the system adjusts the final score accordingly. The model is trained using Unsupervised Contrastive Learning, where it learns to distinguish between an agent's response and its own conversation theme (positive pair) versus themes from unrelated conversations (negative pair).

3. Experimental Evaluation

3.1 Datasets and Attack Categories

The framework was evaluated across six datasets representing three primary attack vectors:

  • Prompt Attacks: Manipulating system prompts to degrade performance (CSQA, MMLU, GSM8K).
  • Tool Attacks: Leveraging external tools for malicious use, such as data theft (InjecAgent).
  • Memory Attacks: Injecting false historical records to disrupt reasoning (PoisonRAG, CSQA).

3.2 Performance Results

XG-Guard consistently outperformed existing unsupervised methods like BlindGuard and DOMINANT. Key findings include:

  • High Detection Accuracy: The model achieved an Area Under the Curve (AUROC) of over 90% in most scenarios. On the InjecAgent dataset, it reached a near-perfect score of 99.56%.
  • Supervised-Level Performance: Despite being unsupervised, XG-Guard performed nearly as well as G-Safeguard, a supervised model that requires manually labeled attack data.
  • Robustness: While "no-defense" systems saw performance plummet as misinformation propagated through the network, XG-Guard maintained high accuracy even as conversations grew in complexity.

4. Explainability and Heatmaps

A core feature of XG-Guard is its ability to provide human-readable explanations. By weighting the contribution of individual tokens to the overall anomaly score, the system generates heatmaps. This allows security engineers to see exactly which words (e.g., "send to external address") triggered the alert, transforming the "black box" into a transparent diagnostic tool.

5. Limitations and Future Work

While effective, XG-Guard faces challenges with spurious tokens. The underlying text encoder (SentenceBERT) can sometimes embed contextual noise, causing the model to highlight punctuation or common words as anomalous. Future research will focus on reducing this noise, exploring broader task domains like open-ended question-answering, and developing methods to automatically update the detector as backend LLMs evolve.

6. Conclusion

XG-Guard represents a significant step forward in securing LLM multi-agent systems. By combining fine-grained token analysis with graph-aware context and dynamic theme modeling, it provides a robust, unsupervised, and explainable defense against the increasingly sophisticated landscape of adversarial attacks.

Summary updated

BibTeX
@inproceedings{pan2026explainable,
  author = {Junjun Pan and Yixin Liu and Rui Miao and Kaize Ding and Yu Zheng and Quoc Viet Hung Nguyen and Alan Wee-Chung Liew and Shirui Pan},
  title = {{Explainable and Fine-Grained Safeguarding of LLM Multi-Agent Systems via Bi-Level Graph Anomaly Detection}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.1407},
}
Figure