Explainable and Fine-Grained Safeguarding of LLM Multi-Agent Systems via Bi-Level Graph Anomaly Detection

Junjun Pan, Yixin Liu, Rui Miao, Kaize Ding, Yu Zheng, Quoc Viet Hung Nguyen, Alan Wee-Chung Liew, Shirui Pan

TL;DR

XG-Guard is a novel unsupervised framework designed to protect Large Language Model (LLM) multi-agent systems from malicious actors. By analyzing communication at both the sentence and token levels, it identifies subtle "camouflaged" attacks and provides human-readable explanations for why specific agents were flagged as anomalies.

What Problem Does This Paper Address?

As LLM-based Multi-Agent Systems (MAS) take on more autonomous, safety-critical roles—from collaborative coding to social simulation—they become prime targets for adversarial attacks. Malicious agents can disrupt these systems via prompt injection, memory poisoning, or tool exploitation.

Existing defenses often rely on Graph Anomaly Detection (GAD), treating the agents as nodes in a communication network. However, these methods suffer from two major flaws:

  1. Coarse Granularity: They typically compress an agent's entire response into a single sentence-level vector. This "averaging" effect often misses subtle, fine-grained malicious cues (like a single injected command) hidden within an otherwise benign response.
  2. Lack of Interpretability: Most GAD models act as black boxes. They might flag an agent as "malicious," but they can't explain which part of the agent's dialogue triggered the alarm, making them difficult to trust in real-world deployments.

Methods

The authors introduce XG-Guard, which operates on a "detect-then-remediate" principle without requiring any labeled examples of attacks (unsupervised). Its architecture consists of three main pillars:

  • Bi-Level Agent Encoder: Unlike previous models, XG-Guard uses a dual-stream architecture. It captures sentence-level representations (holistic meaning) and token-level representations (fine-grained lexical cues). It then uses Graph Neural Networks (GNNs) to incorporate the communication topology—essentially "who is talking to whom"—to enrich these representations.
  • Theme-Based Anomaly Detector: Since "normal" behavior changes depending on the task, XG-Guard dynamically calculates a "theme prototype" for each conversation. It identifies anomalies by measuring how far an agent's behavior deviates from the central theme of the current discussion.
  • Bi-Level Score Fusion & Explanation: To prevent the noisy token-level data from contradicting the sentence-level data, the model uses a covariance-based fusion mechanism. This not only produces a final anomaly score but also allows the model to "back-project" the suspicion onto specific tokens, highlighting exactly which words or phrases (e.g., "send to external email") led to the detection.

The model is trained using contrastive learning, where it learns to align an agent's embedding with its own conversation's theme while distancing it from the themes of unrelated conversations.

Datasets

The framework was evaluated across six diverse datasets covering different attack vectors:

  • Prompt Attacks: CSQA, MMLU, and GSM8K (where system prompts are manipulated to downgrade performance).
  • Tool Attacks: InjecAgent (where agents are tricked into using external tools for malicious purposes, like data theft).
  • Memory Attacks: PoisonRAG and CSQA (where false conversational records are injected to mislead the group).

The researchers also tested these across four communication topologies: Chain, Tree, Star, and Random graphs.

Key Insights and Ideas

  • Subtlety is the Attacker's Shield: The paper highlights that malicious behavior is often "camouflaged" within a small fraction of tokens. By moving to a bi-level representation, the model can "zoom in" on these subtle signals that sentence-level models overlook.
  • Context is King: A static definition of "normal" doesn't work for LLMs. By using dynamic theme prototypes, XG-Guard adapts to the specific topic of the conversation, reducing false positives.
  • Explainability as a Security Feature: Providing token-level heatmaps isn't just a "nice-to-have"; it allows human operators to verify if a detection is a true attack or a misunderstanding, which is critical for system reliability.

Key Results

  • Superior Detection: XG-Guard consistently achieved over 90% AUROC across nearly all test scenarios, significantly outperforming existing unsupervised baselines like BlindGuard.
  • Competitive with Supervised Models: Despite being unsupervised (no labeled attack data), XG-Guard's performance was remarkably close to G-Safeguard, a state-of-the-art supervised model.
  • Robustness: The system maintained high accuracy even as the number of dialogue turns increased, whereas undefended systems saw rapid performance degradation under memory attacks.
  • LLM Agnostic: The framework proved effective across different backbone models, including GPT-4o-mini, DeepSeek-V3, and Qwen-30B.

Limitations and Future Work

  • Spurious Explanations: The authors noted that the model occasionally highlights "spurious tokens" like punctuation marks. This happens because the pre-trained text encoders (like SentenceBERT) are black boxes that sometimes bleed contextual information into nearby tokens.
  • Scope of Tasks: While the model excels at the tested benchmarks, the authors suggest future work should explore even broader task domains, such as real-world complex decision-making and open-ended question-answering.
  • API Stability: Since the system relies on LLM backbones, updates to the underlying APIs by providers could potentially impact the stability of the anomaly detector, a challenge common to many LLM-based tools.

Summary updated

Abstract

Large language model (LLM)-based multi-agent systems (MAS) have shown strong capabilities in solving complex tasks. As MAS become increasingly autonomous in various safety-critical tasks, detecting malicious agents has become a critical security concern. Although existing graph anomaly detection (GAD)-based defenses can identify anomalous agents, they mainly rely on coarse sentence-level information and overlook fine-grained lexical cues, leading to suboptimal performance. Moreover, the lack of interpretability in these methods limits their reliability and real-world applicability. To address these limitations, we propose XG-Guard, an explainable and fine-grained safeguarding framework for detecting malicious agents in MAS. To incorporate both coarse and fine-grained textual information for anomalous agent identification, we utilize a bi-level agent encoder to jointly model the sentence- and token-level representations of each agent. A theme-based anomaly detector further captures the evolving discussion focus in MAS dialogues, while a bi-level score fusion mechanism quantifies token-level contributions for explanation. Extensive experiments across diverse MAS topologies and attack scenarios demonstrate robust detection performance and strong interpretability of XG-Guard.
BibTeX
@inproceedings{pan2026explainable,
  author = {Junjun Pan and Yixin Liu and Rui Miao and Kaize Ding and Yu Zheng and Quoc Viet Hung Nguyen and Alan Wee-Chung Liew and Shirui Pan},
  title = {{Explainable and Fine-Grained Safeguarding of LLM Multi-Agent Systems via Bi-Level Graph Anomaly Detection}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.1407},
}

Loading related content...

Figure