ReachAgent: Enhancing Mobile Agent via Page Reaching and Operation

Qinzhuo Wu, Wei Liu, Jian Luan, Bin Wang

Abstract

Recently, mobile AI agents have gained increasing attention for their ability to interact with mobile devices across multiple steps to solve complex tasks. However, existing agents often focus on the most task-relevant elements at each individual step, leading to local optimal solutions that ignore the overall GUI flow. To address this, we introduce ReachAgent, a two-stage framework that decomposes tasks into "page reaching" and "page operation" subtasks. By utilizing a new training dataset, MobileReach, and a reward-based preference system, ReachAgent significantly improves task-completion abilities. Experimental results demonstrate that ReachAgent outperforms state-of-the-art (SOTA) agents, improving step-level IoU and Text Accuracy by 7.12% and 7.69%, respectively.

Introduction

Current mobile AI agents—models designed to navigate applications like a human user—frequently suffer from a "greedy" search problem. When tasked with a multi-step instruction (e.g., "Add a white Xiaomi 14 to my cart"), these agents tend to search for the most task-relevant keywords on the current screen. If the agent is on a homepage and does not immediately see the target item, it may click irrelevant banners or become stuck in a loop because it fails to recognize the necessity of a "Reach" subtask, such as using a search bar or navigating categories.

Existing agents prioritize single-step accuracy but often overlook the global GUI flow required to complete a multi-step job. ReachAgent addresses this by explicitly training the model to distinguish between navigating to a destination and performing operations once the destination is reached.

The ReachAgent Framework

ReachAgent is built on a MobileVLM backbone and utilizes a two-stage training process to refine its navigation and interaction logic.

Stage 1: Supervised Fine-Tuning (SFT)

The model is first trained on the MobileReach dataset to acquire three distinct skills:

  1. General Navigation: Understanding basic app structures.
  2. Page Reaching: Learning the sequences required to arrive at specific functional pages.
  3. Page Operation: Executing specific tasks (e.g., selecting attributes, clicking "buy") once the correct page is reached.

Stage 2: Reinforcement Learning via DPO

To improve efficiency, the framework employs Direct Policy Optimization (DPO) using a 4-level reward function to rank GUI flows:

  • Golden: The most efficient, shortest path to success.
  • Longer: A successful path that includes unnecessary steps.
  • Incomplete: A path that failed to reach the goal.
  • Invalid: A path containing impossible actions, such as clicking non-existent UI elements.

Action Alignment Mechanism

A key technical innovation in ReachAgent is the Action Alignment Mechanism. Rather than predicting arbitrary X/Y coordinates on a raw screenshot, ReachAgent targets the center points of UI elements identified in the application's XML code. This reduces the search space and ensures the agent interacts with valid, functional components of the interface.

Datasets: MobileReach

The researchers developed MobileReach, a comprehensive dataset derived from "Mobile3M" covering 49 popular apps. The dataset includes:

  • Page Navigation: ~54k flows for moving between screens.
  • Page Reaching: ~68k flows focused on reaching specific destinations.
  • Page Operation: ~76k flows for executing tasks on target pages.

Additionally, the model was evaluated on the Auto-UI benchmark to test its generalization capabilities on unseen tasks.

Results and Insights

ReachAgent demonstrates a significant performance leap over previous SOTA mobile agents:

  • Step-Level Accuracy: Improved Intersection over Union (IoU) by 7.12% and Text Accuracy by 7.69%.
  • Task-Level Success: Achieved a 4.72% and 4.63% boost in completing entire multi-step flows correctly.
  • Generalization: Even on the Auto-UI dataset, where it was not specifically trained on subtasks, ReachAgent outperformed all baselines, proving that the "Reach and Operate" logic is a universally effective strategy for mobile interaction.

Conclusion

ReachAgent highlights that subtask decomposition is critical for mobile AI. By teaching the model that reaching a page is a valid intermediate goal, the framework prevents "greedy" errors. Furthermore, by favoring "Golden" paths through DPO, ReachAgent learns to be an efficient "power user" rather than just a functional one. Future work will focus on automated exploration to build deeper datasets and further refining reward models to capture nuanced user preferences.

Summary updated

Abstract

Recently, mobile AI agents have gained increasing attention. Given a task, mobile AI agents can interact with mobile devices in multiple steps and finally form a GUI flow that solves the task. However, existing agents tend to focus on most task-relevant elements at each step, leading to local optimal solutions and ignoring the overall GUI flow. To address this issue, we constructed a training dataset called MobileReach, which breaks the task into page reaching and operation subtasks. Furthermore, we propose ReachAgent, a two-stage framework that focuses on improving its task-completion abilities. It utilizes the page reaching and page operation subtasks, along with reward-based preference GUI flows, to further enhance the agent. Experimental results show that ReachAgent significantly improves the IoU Acc and Text Acc by 7.12% and 7.69% on the step-level and 4.72% and 4.63% on the task-level compared to the SOTA agent. Our data and code will be released upon acceptance.
BibTeX
@inproceedings{wu2025reachagent,
  author = {Qinzhuo Wu and Wei Liu and Jian Luan and Bin Wang},
  title = {{ReachAgent: Enhancing Mobile Agent via Page Reaching and Operation}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2025},
  url = {https://openreview.net/forum?id=U4rtPpgjLz},
}
Figure