OS-Sentinel: Towards Safety-Enhanced Mobile GUI Agents via Hybrid Validation in Realistic Workflows

Qiushi Sun, Mukai Li, Zhoumianze Liu, Zhihui Xie, Fangzhi Xu, Zhangyue Yin, Kanzhi Cheng, Zehao Li, Zichen Ding, Qi Liu, Zhiyong Wu, Zhuosheng Zhang, Ben Kao, Lingpeng Kong

TL;DR

While Vision-Language Model (VLM) agents excel at automating mobile tasks, they pose severe safety risks like privacy leaks and system compromise—even when executing completely benign user instructions. To address this, researchers have introduced OS-Sentinel, a hybrid framework that pairs a formal system-level verifier with a VLM contextual judge, boosting safety detection by 10% to 30% over standard baselines. The framework is supported by MobileRisk-Live, a new dynamic Android sandbox and benchmark for reproducible agent safety research.

What Problem Does This Paper Address?

Autonomous mobile GUI agents can navigate apps, click buttons, and type text just like humans. However, their autonomy introduces massive security liabilities. A seemingly innocent user request can inadvertently trigger unsafe agent behaviors, such as leaking sensitive medical data or posting offensive content.

Figure 1: A normal user request can trigger unexpected safety issues in mobile agents, such as privacy violations and socially offensive behaviors.

Current safety guardrails fall into two flawed camps:

  1. Deterministic rule-checkers: These look for specific keywords or UI elements but fail to scale and lack the semantic context to separate benign actions from true violations.
  2. LLM/VLM-as-a-Judge methods: These evaluate screenshots or accessibility trees but are completely blind to underlying system-level changes (like background file tampering or unauthorized permission escalation) and lack auditability.

Furthermore, existing safety benchmarks mostly focus on desktop or web environments, leaving the highly sensitive and dynamic mobile domain largely underexplored.

Methods

The authors propose OS-Sentinel, a hybrid safety detection framework that operates at both the individual step level (for real-time intervention) and the full trajectory level (for post-hoc analysis).

Figure 4: Illustration of OS-Sentinel, a hybrid safety detection framework. At each step, agent actions, GUI observations, and system state traces are jointly assessed by a contextual judge and a formal verifier to determine safety. The framework supports safety detection for mobile GUI agents at both the step level and the trajectory level.

OS-Sentinel splits the safety evaluation into two specialized components:

  • Formal Verifier (Unified Rule-based System Checker): Instead of checking the UI, it monitors a hidden System State Trace ($T_{sys}$) extracted from the Android Runtime. It uses three mechanisms:
  • System State Integrity Monitoring: Computes cryptographic hashes (SHA256) over file system metadata. Any unexpected hash mismatch flags unauthorized background configuration changes or privilege escalation.
  • Sensitive Keyword Detection: Performs exact string matching against a curated lexicon of financial, personal, and security terms.
  • Sensitive Pattern Matching: Uses regular expressions to detect structured data like credit card numbers, passwords, and email addresses.
  • Contextual Judge (Model-based Safety Analysis): A VLM or LLM evaluates the semantic meaning of the agent's actions. It reviews sequences of screenshots or accessibility trees across a trajectory to infer behavioral intent and catch contextual risks (like social engineering or compliance violations) that rules alone cannot detect.

The framework combines these insights into a Hybrid Verdict using a configurable aggregation function. By default, it operates in a "strict mode" (logical OR), flagging a step as unsafe if either the verifier or the judge raises an alarm.

Datasets

To evaluate mobile agent safety, the authors built two core resources:

  1. MobileRisk-Live: A dynamic Android emulator sandbox packaged with 50 pre-installed applications and system components. It records standard GUI observations alongside background system state traces ($T_{sys}$) and supports automated state restoration.
  2. MobileRisk: A benchmark dataset consisting of 204 fine-grained agent trajectories (102 safe, 102 unsafe). The unsafe trajectories span a robust taxonomy of 10 risk categories, divided into user-side risks (e.g., prompt injection) and agent-side risks (e.g., over-automation, resource abuse).
Figure 2: MobileRisk-Live provides a dynamic Android emulator sandbox that supports realistic GUI agent interactions and state restoration. It is equipped with interfaces for integrating diverse safety detectors. MobileRisk is constructed from frozen trajectories collected in this environment, where each step contains a screenshot, a11ytree, system state trace, and action. These trajectories are further annotated at multiple granularities.

Key Insights and Ideas

  • The GUI Layer Lies: Many critical safety violations happen entirely in the background. For instance, an agent might modify a system XML configuration file—an action completely invisible on the screen or in the accessibility tree, but easily caught by tracking file system metadata hashes.
  • Asymmetry of Error Costs: In real-world deployment, a false positive (blocking a safe action) merely pauses execution for human review, whereas a false negative (missing a malicious action) can cause catastrophic data loss. OS-Sentinel deliberately prioritizes safety recall over precision.
  • Synergy Wins: Rule-based verifiers and VLM judges have complementary blind spots. The formal verifier excels at detecting explicit system violations, while the VLM judge handles nuanced semantic context. Combining them yields a much safer operational boundary than scaling up either component individually.

Key Results

OS-Sentinel consistently outperformed rule-based and model-only baselines across multiple VLM backbones (including GPT-4o, Claude-3.7-Sonnet, and Qwen2.5-VL).

  • Trajectory-Level Performance: When using the Claude-4.5-Sonnet backbone, OS-Sentinel achieved 71.7% accuracy and a 73.0% F1-score, compared to just 60.2% accuracy and 57.1% F1 for the standalone VLM-as-a-Judge baseline.
  • Component Analysis: Ablation tests confirmed that the hybrid combination significantly boosts both accuracy and F1-scores across different model scales, proving that the verifier and judge successfully cover for each other's weaknesses.
Figure 5: Trajectory-level component analysis across three backbones (Accuracy).
  • Real-World Feasibility: For API-based models, OS-Sentinel demonstrated an average latency of just 66ms per detection, making it highly practical for real-time deployment as an interactive guardrail.

Limitations and Future Work

  • Closed Ecosystem Restrictions: The Formal Verifier relies on gathering system state traces. While these are accessible on open platforms like Android via standard APIs without root access, this approach cannot be easily ported to closed operating systems like iOS.
  • The False Positive Trade-off: Because the framework's strict mode enforces a zero-tolerance policy, it frequently flags benign user-requested maintenance tasks (such as "clear app cache to free up space") as destructive system modifications. The authors note that aligning the framework's sensitivity with the subjective, nuanced tolerance of human users remains an open challenge for future research.

Summary updated

Abstract

Computer-using agents powered by Vision-Language Models (VLMs) have demonstrated human-like capabilities in operating digital environments like mobile platforms. While these agents hold great promise for advancing digital automation, their potential for unsafe operations, such as system compromise and privacy leakage, is raising significant concerns. Detecting these safety concerns across the vast and complex operational space of mobile environments presents a formidable challenge that remains critically underexplored. To establish a foundation for mobile agent safety research, we introduce MobileRisk-Live, a dynamic sandbox environment accompanied by a safety detection benchmark comprising realistic trajectories with fine-grained annotations. Built upon this, we propose OS-Sentinel, a novel hybrid safety detection framework that synergistically combines a Formal Verifier for detecting explicit system-level violations with a VLM-based Contextual Judge for assessing contextual risks and agent actions. Experiments show that OS-Sentinel achieves 10%-30% improvements over existing approaches across multiple metrics. Further analysis provides critical insights that foster the development of safer and more reliable autonomous mobile agents. Our code and data are available at https://qiushisun.github.io/OS-Sentinel-Home/.
BibTeX
@inproceedings{sun2026os,
  author = {Qiushi Sun and Mukai Li and Zhoumianze Liu and Zhihui Xie and Fangzhi Xu and Zhangyue Yin and Kanzhi Cheng and Zehao Li and Zichen Ding and Qi Liu and Zhiyong Wu and Zhuosheng Zhang and Ben Kao and Lingpeng Kong},
  title = {{OS-Sentinel: Towards Safety-Enhanced Mobile GUI Agents via Hybrid Validation in Realistic Workflows}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.431},
}
Figure