FinVault: Benchmarking Financial Agent Safety in Execution-Grounded Environments

Zhi Yang, Runguo Li, Qiqi Qiang, Jiashun Wang, Fangqi Lou, Mengping Li, Dongpo Cheng, Rui Xu, Heng Lian, Shuo Zhang, Xiaolong Liang, Xiaoming Huang, Zheng Wei, Zhaowei Liu, Xin Guo, Huacan Wang, Ronghao Chen, Liwen Zhang

TL;DR

This paper introduces FinVault, the first benchmark for evaluating the security of financial AI agents in realistic, execution-grounded environments. It reveals that current safety measures are insufficient, with even advanced models susceptible to attacks that can lead to significant financial losses and regulatory violations.

What Problem Does This Paper Address?

Financial AI agents, powered by Large Language Models (LLMs), are increasingly used for critical tasks like investment analysis and risk assessment. These agents can plan, use tools, and manage state, which introduces new security risks in the highly regulated and high-stakes financial sector. Existing safety evaluations often focus on the LLM's text output or abstract agent settings, failing to capture the real-world risks that arise from these agents executing actions and changing system states. This gap means we don't truly know how safe these agents are when they interact with actual financial systems, potentially leading to financial losses and regulatory breaches.

Methods

The core contribution is FinVault, a novel security benchmark designed to evaluate financial agents in "execution-grounded" environments. This means the agents operate within simulated, but realistic, financial workflows. Key aspects of FinVault include:

  • Regulation-Driven Scenarios: 31 scenarios are based on real regulatory violations and case studies, ensuring practical relevance.
  • State-Writable Databases: These scenarios include databases that agents can interact with and modify, mimicking real financial systems.
  • Explicit Compliance Constraints: The environments enforce specific rules and regulations that agents must adhere to.
  • Vulnerability-Driven Threat Models: The benchmark incorporates 107 real-world vulnerabilities, categorized into types like privilege bypass, compliance violations, and information leakage.
  • Attack Taxonomy: A comprehensive set of 8 attack techniques is used, including prompt injection, jailbreaking, and financially adapted attacks.
  • Sandboxed Environments: Agents operate in isolated environments to safely test their security without real-world consequences.

Datasets

The paper introduces FinVault, a new benchmark dataset. It comprises:

  • 31 regulatory case-driven financial business scenarios.
  • 107 predefined high-risk vulnerabilities.
  • 963 test cases, including 856 adversarial attack samples (across 8 attack techniques and 107 vulnerabilities) and 107 benign business samples for false positive evaluation.

Key Insights and Ideas

  • General Safety Doesn't Transfer: Standard LLM safety alignment techniques are not sufficient for financial agents. The complex, ambiguous rules in finance create unique vulnerabilities that general defenses miss.
  • Semantic Attacks Dominate: Attacks that manipulate the agent's understanding and reasoning (like role-playing or emotional manipulation) are far more effective than purely technical attacks (e.g., encoding obfuscation).
  • Instruction Isolation is Crucial: How well an agent separates system instructions from user inputs significantly impacts its vulnerability to attacks like instruction overriding. Claude models, for instance, show much higher resistance due to stricter separation.
  • Context is King (for Attackers): Framing requests as "test mode," "academic discussion," or "emergency situations" can drastically lower an agent's security vigilance.
  • Multi-Turn Risk: A single-turn defense is not enough. Attacks can progressively compromise an agent over multiple interactions, especially through methods like emotional manipulation.
  • Insurance is Riskiest: Insurance scenarios exhibit the highest attack success rates, likely due to their reliance on complex policy interpretation and discretionary decision-making.

Key Results

  • High Attack Success Rates: Even state-of-the-art models show significant vulnerabilities. The most vulnerable models have an average Attack Success Rate (ASR) of up to 50.0%, while even the most robust models still have exploitable vulnerabilities in 20.56% of cases.
  • Defense Ineffectiveness: Existing defense mechanisms are largely ineffective. They often struggle with semantic attacks and face a trade-off between detection capability and false positives.
  • LLaMA Guard 4 Performance: Among the evaluated defense methods, LLaMA Guard 4 showed the highest True Positive Rate (TPR) at 61.10%, but also a notable False Positive Rate (FPR) of 29.91%. GPT-OSS-Safeguard had the lowest FPR (12.15%) but a much lower TPR (22.07%).

Limitations and Future Work

  • Limited Attack Techniques: While FinVault covers dominant attack paradigms, emerging strategies are not yet included.
  • Sandbox vs. Production: The sandboxed environments approximate real workflows but inevitably differ in scale, complexity, and operational constraints from production systems.
  • Uniform ASR Metric: The ASR metric treats all successful attacks equally, not accounting for differences in severity or downstream impact.

The authors aim to support the community in developing, evaluating, and deploying safer financial AI agents. Future work could involve expanding the attack taxonomy, incorporating more complex and dynamic environments, and developing more nuanced evaluation metrics.

Summary updated

Abstract

Financial agents powered by large language models (LLMs) are increasingly deployed for investment analysis, risk assessment, and automated decision-making, where their abilities to plan, invoke tools, and manipulate mutable state introduce new security risks in high-stakes and highly regulated financial environments. However, existing safety evaluations largely focus on language-model-level content compliance or abstract agent settings, failing to capture execution-grounded risks arising from real operational workflows and state-changing actions. To bridge this gap, we propose FinVault, the first execution-grounded security benchmark for financial agents, comprising 31 regulatory case-driven sandbox scenarios with state-writable databases and explicit compliance constraints, together with 107 real-world vulnerabilities and 963 test cases that systematically cover prompt injection, jailbreaking, financially adapted attacks, as well as benign inputs for false-positive evaluation. Experimental results reveal that existing defense mechanisms remain ineffective in realistic financial agent settings, with average attack success rates (ASR) still reaching up to 50.0\% on state-of-the-art models and remaining non-negligible even for the most robust systems (ASR 6.7\%), highlighting the limited transferability of current safety designs and the need for stronger financial-specific defenses. Our code can be found at https://github.com/aifinlab/FinVault.
BibTeX
@misc{yang2026finvault,
  author = {Zhi Yang and Runguo Li and Qiqi Qiang and Jiashun Wang and Fangqi Lou and Mengping Li and Dongpo Cheng and Rui Xu and Heng Lian and Shuo Zhang and Xiaolong Liang and Xiaoming Huang and Zheng Wei and Zhaowei Liu and Xin Guo and Huacan Wang and Ronghao Chen and Liwen Zhang},
  title = {{FinVault: Benchmarking Financial Agent Safety in Execution-Grounded Environments}},
  year = {2026},
  eprint = {2601.07853},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2601.07853},
}
Figure