AgentProg: Empowering Long-Horizon GUI Agents with Program-Guided Context Management

Shizuo Tian, Hao Wen, Yuxuan Chen, Jiacheng Liu, Shanhui Zhao, Guohong Liu, Ju Ren, Yunxin Liu, Yuanchun Li

Abstract

Large Language Model (LLM)-based agents for Graphical User Interfaces (GUIs) often fail during long-horizon tasks due to context window limitations and the loss of critical information over time. We introduce AgentProg, a framework that treats agent interaction history as a structured program. By utilizing a "fuzzy" domain-specific language and a global belief state mechanism, AgentProg manages context efficiently, enabling agents to complete complex, multi-step tasks across mobile applications.

Introduction

Building intelligent agents capable of navigating mobile applications is a cornerstone of autonomous AI. However, current agents hit a "context wall" when tasks extend over dozens or hundreds of steps. Standard approaches suffer from context explosion, where history consumes excessive tokens; information loss, where critical details are discarded by compression; and partial observability, where agents lose track of the environment's internal state.

Common failure modes in standard GUI agents, including planning gaps, information loss, and environmental misunderstanding.

Methodology: The Program-Guided Approach

AgentProg reframes task execution as a running computer program, employing three core components to maintain focus and efficiency.

Semantic Task Program (STP)

The STP is a domain-specific language (DSL) that bridges the gap between rigid code and flexible natural language. It uses programming constructs like loops (iterate over {task_list}), conditionals (if {age} < 18), and variables (record as {phone_number}) while keeping the underlying instructions in "soft" natural language. This allows the agent to follow a logical plan while adapting to UI variations.

Program-Guided Context Management

Instead of maintaining a flat action history, AgentProg organizes interactions into an Execution Tree. It employs Path-Based Pruning to remove irrelevant branches (e.g., completed loop iterations) and uses Variables as persistent anchors for critical data, ensuring information remains accessible regardless of task length.

Global Belief State

Inspired by the Belief MDP framework, this module maintains a "mental model" of the environment through a Predict-Verify-Align cycle. The agent predicts the expected UI state, verifies it against actual screenshots, and triggers recovery routines if a "Belief-Reality Gap" (such as an app crash) is detected.

Experimental Evaluation

The researchers evaluated AgentProg on AndroidWorld (116 tasks) and a new, more challenging suite called AW-Extend. AW-Extend focuses on long-horizon capabilities, including compositional multi-app workflows and iterative tasks requiring repeated sub-task execution.

Example of a complex compositional task involving calendar events, contact creation, and expense management.

Key Results

AgentProg demonstrates significant improvements in both success rate and efficiency:

  • State-of-the-Art Performance: Achieved a 78.0% success rate on AndroidWorld, a 4.7% improvement over previous benchmarks.
  • Long-Horizon Robustness: On the AW-Extend benchmark, AgentProg reached 68.4%, nearly doubling the performance of the next best baseline (36.8%).
  • Context Efficiency: Unlike standard agents whose memory usage grows linearly, AgentProg maintains a nearly constant dynamic context length (~9k tokens) even during 50+ step tasks.
  • Ablation Insights: Removing the Global Belief State caused the success rate to drop from 78% to 53.9%, proving the necessity of active environment monitoring.

Conclusion and Future Work

AgentProg demonstrates that treating interaction history as a structured program is a powerful paradigm for long-horizon GUI agents. While the current implementation faces challenges regarding latency and prompt overhead, future work involving fine-tuned specialized models for STP generation could further optimize performance and responsiveness.

Summary updated

Abstract

The rapid development of mobile GUI agents has stimulated growing research interest in long-horizon task automation. However, building agents for these tasks faces a critical bottleneck: the reliance on ever-expanding interaction history incurs substantial context overhead. Existing context management and compression techniques often fail to preserve vital semantic information, leading to degraded task performance. We propose AgentProg, a program-guided approach for agent context management that reframes the interaction history as a program with variables and control flow. By organizing information according to the structure of program, this structure provides a principled mechanism to determine which information should be retained and which can be discarded. We further integrate a global belief state mechanism inspired by Belief MDP framework to handle partial observability and adapt to unexpected environmental changes. Experiments on AndroidWorld and our extended long-horizon task suite demonstrate that AgentProg has achieved the state-of-the-art success rates on these benchmarks. More importantly, it maintains robust performance on long-horizon tasks while baseline methods experience catastrophic degradation. Our system is open-sourced at https://github.com/MobileLLM/AgentProg.
BibTeX
@inproceedings{tian2026agentprog,
  author = {Shizuo Tian and Hao Wen and Yuxuan Chen and Jiacheng Liu and Shanhui Zhao and Guohong Liu and Ju Ren and Yunxin Liu and Yuanchun Li},
  title = {{AgentProg: Empowering Long-Horizon GUI Agents with Program-Guided Context Management}},
  booktitle = {MobiSys},
  year = {2026},
  doi = {10.1145/3745756.3809245},
}
Figure