GuirlVG: Incentivize GUI Visual Grounding via Empirical Exploration on Reinforcement Learning

Weitai Kang, Bin Lei, Gaowen Liu, Caiwen Ding, Yan Yan

TL;DR

GuirlVG introduces a highly data-efficient reinforcement learning framework for Graphical User Interface Visual Grounding (GUI-VG) that shatters the reliance on massive supervised datasets. By systematically optimizing Group Relative Policy Optimization (GRPO) with a novel Soft Reward Function and an Adversarial KL Factor, GuirlVG achieves state-of-the-art performance using just 5.2K training samples—outperforming traditional supervised methods trained on over 10 million samples.

Figure 1: Step-by-step exploration of GuirlVG. Starting from trivial RFT, we progressively add Soft Reward Function, In-Bbox reward with point prediction, β tuning, our Adversarial KL Factor, image resolution prompting, and extended training. With only 5.2K data, GuirlVG surpasses SFT methods trained on up to 13.58M data. Circle size reflects data scale used by each method.

What Problem Does This Paper Address?

For GUI agents to interact with digital environments, they must excel at GUI Visual Grounding (GUI-VG)—the ability to locate actionable elements (like buttons or icons) on a screenshot based on textual instructions. Historically, achieving high accuracy required Supervised Fine-Tuning (SFT) on millions of meticulously annotated screenshots, incurring massive data curation and training costs.

As Multimodal Large Language Models (MLLMs) advance, they inherently absorb GUI knowledge during pretraining, making exhaustive post-training SFT increasingly redundant and inefficient. While rule-based reinforcement fine-tuning (RFT) has emerged as a cheaper alternative in reasoning domains, a naive application of RFT to GUI tasks actually underperforms basic SFT. This paper tackles a fundamental question: How do we optimally formulate reinforcement learning objectives to unlock data-efficient GUI visual grounding?

Methods

The authors introduce GuirlVG, built on top of the GRPO algorithm. Instead of accepting standard RFT defaults, they systematically re-engineered its core components:

  1. Soft Reward Function (SRF): Standard RFT strictly penalizes formatting errors (e.g., missing a JSON bracket or a closing tag), which introduces training instability. SRF relaxes these constraints by offering partial credit for tag presence and ignoring minor stylistic deviations, focusing purely on the extracted coordinates.
  2. Functional Goal Alignment: Rather than forcing the model to predict bounding boxes (Bbox) and calculating Intersection-over-Union (IoU), GuirlVG switches to direct point prediction paired with a binary In-Bbox reward (checking if the point falls inside the ground-truth box).
  3. Adversarial KL Factor: To prevent the model from over-optimizing or "reward hacking" high-value responses, the authors introduce a dynamic KL penalty modifier ($\alpha_i$). Instead of a static penalty, $\alpha_i$ scales proportionally to the achieved reward, adaptively strengthening regularization as rewards increase.
  4. Strategic Prompting: The framework strategically withholds image resolution information during training to force the model to learn robust spatial reasoning, only providing it at test time to refine final predictions.

Datasets

  • Training: 5,200 samples randomly selected from ShowUI (a dataset containing visually rich website data and desktop data from OmniAct augmented via GPT-4o).
  • Evaluation: Evaluated across three major GUI-VG benchmarks:
  • ScreenSpot: Evaluates grounding across mobile, desktop, and web environments.
  • ScreenSpot v2: An updated version of ScreenSpot with corrected annotation errors.
  • ScreenSpot-Pro: A challenging benchmark focusing on high-resolution professional applications spanning 23 applications and 5 industries.

Key Insights and Ideas

  • "SFT memorizes, RL generalizes": GuirlVG achieved massive gains on out-of-domain tasks. For instance, it outperformed baselines on mobile screenshots by +11.8% despite containing zero mobile-specific samples in its 5.2K training set.
  • Rigid formatting rewards hurt RL training: Giving partial credit for formatting allows the model to focus its exploration on spatial reasoning rather than getting bogged down by strict JSON syntax.
  • Task-aligned objectives matter: Aligning the reward directly with the evaluation metric (point-in-box accuracy rather than bounding box IoU) yields substantial performance boosts.
  • Withholding context can foster better learning: Counterintuitively, hiding image resolution during training forces the model to build stronger internal spatial awareness, making the resolution cue highly effective when reintroduced at inference.

Key Results

GuirlVG demonstrates unprecedented data efficiency, achieving state-of-the-art results across all evaluated benchmarks:

  • ScreenSpot: Reached 88.7% accuracy with 5.2K samples, outperforming OS-Atlas (trained on 13.58M samples) by +7.7%.
  • ScreenSpot v2: Achieved a new state-of-the-art 91.9% average accuracy.
  • ScreenSpot-Pro: Delivered an absolute gain of +17.2% over the best SFT baseline, proving its robust generalization capabilities in complex, high-resolution professional software environments.

Limitations and Future Work

The authors note a few limitations in their current study:

  • Model Scope: The empirical exploration was restricted to a single MLLM architecture (Qwen2.5-VL), leaving its generalizability across other architectures unverified.
  • Compute & Scale Constraints: Due to hardware limitations, the authors could not experiment with larger model scales (such as 32B or 72B parameter variants), which could yield further insights into how reinforcement learning scales for GUI tasks.

Summary updated

Abstract

Graphical user interface visual grounding (GUI-VG), a core capability for GUI agents, has primarily relied on supervised fine-tuning (SFT) of multimodal large language models (MLLMs), which demands extensive data curation and significant training costs. However, as MLLMs continue to advance and even cover GUI domains during pretraining, the necessity of exhaustive SFT post-training becomes increasingly questionable. Meanwhile, recent successes of rule-based reinforcement fine-tuning (RFT) suggest a more efficient alternative. Despite this promise, the optimal manner of applying RFT for GUI-VG remains unexplored. To bridge this gap, we introduce GuirlVG, a reinforcement learning-based GUI-VG method built on a systematic empirical study and a novel stabilization technique. We find that naive application of RFT underperforms the SFT baseline, motivating a deeper exploration. First, we decompose RFT into its core components and analyze the optimal formulation of each. Second, we propose a novel Adversarial KL Factor that dynamically stabilizes training to mitigate reward over-optimization. Third, we further explore the training configurations of RFT to enhance effectiveness. Extensive experiments show that GuirlVG, with only 5.2K training samples, outperforms SFT methods trained on over 10M samples, achieving a 7.7% improvement on ScreenSpot, a 17.2% improvement on ScreenSpotPro, and 91.9% accuracy on ScreenSpotV2.
BibTeX
@inproceedings{kang2026guirlvg,
  author = {Weitai Kang and Bin Lei and Gaowen Liu and Caiwen Ding and Yan Yan},
  title = {{GuirlVG: Incentivize GUI Visual Grounding via Empirical Exploration on Reinforcement Learning}},
  booktitle = {International Conference on Learning Representations},
  year = {2026},
  url = {https://openreview.net/forum?id=zrH2A1upAo},
}
Figure