Anticipatory Robot Goalkeeping via Monotone Optimal Stopping

Hao E. Zhang, Ruize Geng, Yisen Li, Yaru Niu, Yikai Wang, Raihan Haque, Khalil Zbiss, Guanyang Luo, Hui-ping Wang, H. Eric Tseng, Ding ZhaoarXiv 2026 · arXiv

TL;DR

A robot goalkeeper faces a classic dilemma: act early under high uncertainty, or wait for better data and risk running out of time to reach the ball. This paper introduces Monotone Optimal Stopping (MOS), a framework that treats this "when to act" decision as a finite-horizon optimal stopping problem. By directly learning an act-versus-wait margin constrained by physical urgency, MOS significantly improves overall save rates and excels at adapting to deceptive human feints.

What Problem Does This Paper Address?

In fast-paced physical interactions, robots must often execute movements before they have complete information about their opponent's intent. Goalkeeping is a prime example of this "act-before-certainty" challenge.

As a ball flies toward the goal, a robot's situation changes in two opposing directions:

  1. Information improves: As time passes, the ball's trajectory and destination become clearer.
  2. Physical opportunity shrinks: The longer the robot waits, the less time it has to accelerate and cover distant corners of the goal.
The information-opportunity trade-off in anticipatory goalkeeping.

Existing goalkeeping systems often rely on fixed heuristics or unconstrained learned triggers. However, confidence thresholds ignore whether the robot is physically capable of reaching the target from its current state, while unconstrained learned gates can produce erratic, non-monotonic activation boundaries. The authors tackle the challenge of mathematically structuring this decision so the robot triggers its motion precisely when waiting one more step costs more in lost opportunity than it gains in information.

Methods

The authors propose a two-stage architecture that separates how to save from when to save:

  1. Closed-Loop Motor Policy: First, they train a low-level motor policy using Proximal Policy Optimization (PPO) on a quadruped robot. This policy maps the robot's state and current target belief to joint-position commands. Once trained, this policy is frozen.
  2. Monotone Optimal Stopping (MOS): MOS handles the high-level activation timing. Instead of learning separate value functions for "acting now" and "waiting" and subtracting them (which introduces compounding approximation errors), MOS directly learns the stopping margin using a direct Bellman-Snell recursion.
The MOS pipeline separating information state processing, online execution, and margin learning.

A key mathematical innovation is the imposition of a partial monotonicity constraint. For any given robot state and target belief, an increase in physical urgency (i.e., less time remaining) must monotonically increase the favorability of acting over waiting. This structural prior guarantees a clean threshold-structured release boundary. Crucially, "release" simply switches the robot from a "ready" mode to the "save" mode; because the underlying motor policy is closed-loop, the robot can still dynamically redirect its dive if the target belief changes post-release.

Datasets

The framework was trained and evaluated using simulated environments in Isaac Lab and validated on physical hardware:

  • Simulation Protocols:
  • Protocol P (Physics Probe): 11,151 episodes used to characterize physical reachability and opportunity across various launch speeds (10 to 15 m/s).
  • Protocol A (Timing Benchmark): 2,000 episodes per seed (across 3 seeds) evaluating different timing mechanisms under controlled belief trajectories, specifically including "reversal" episodes where the initial trajectory cue is intentionally misleading (simulating a feint).
  • Protocol C (Chase Evaluation): 601 shots analyzing closed-loop outcomes across different target bands.
  • Physical Hardware: Deployed on a Unitree Go2 quadruped robot tasked with intercepting balls thrown by a human operator executing physical feints.

Key Insights and Ideas

  • Direct Margin Learning: Learning the act-vs-wait margin directly via the Snell envelope avoids the high variance and approximation errors associated with subtracting two independently fitted neural networks at deployment.
  • Urgency as a Monotonic Prior: Imposing monotonicity only along the physical urgency dimension provides strong geometric guarantees (preventing irregular wait-act-wait behavior) without over-constraining the neural network's ability to interpret complex, non-monotonic target beliefs or robot states.
  • Mode Switching vs. Target Commitment: Activating the save policy is a one-way trigger for the physical effort, but not a final commitment to a spatial target. Because the underlying policy continuously processes updated beliefs, early physical release preserves reachability without sacrificing the ability to adapt to late-stage trajectory changes.

Key Results

  • Higher Save Rates: In extensive simulation studies, MOS achieved a 74.4% mean save rate, a notable improvement over a parameter-matched unconstrained learned gate (67.7%).
  • Robustness to Feints: On "Reversal" shots—where the ball's apparent direction changes mid-flight—MOS boosted the save rate from 52.1% to 66.5%, demonstrating its ability to initiate early motion without getting locked into an incorrect initial prediction.
  • Hardware Validation: Real-robot experiments successfully demonstrated rapid lateral interception and mid-flight direction corrections when facing human shot-direction feints.

Limitations and Future Work

  • Policy-Conditional Limitation: The optimal stopping decision is strictly dependent on a fixed, pre-trained motor policy. Jointly optimizing the timing mechanism and the motor policy remains an open, computationally challenging problem.
  • One-Way Trigger Architecture: The current formulation assumes a single, irreversible switch from "ready" to "save" mode per episode. While ideal for a goalkeeper defending a single shot, tasks requiring repetitive or cyclical act-wait behavior would require extending this framework to multi-stopping or sequential control problems.

Summary updated

Abstract

Robots engaged in fast physical interactions often need to act before the intent of another agent is fully known. Anticipatory goalkeeping illustrates this challenge. Waiting provides more reliable information about the target but reduces the physical opportunity for interception, whereas acting early preserves reachability but requires initiating motion under uncertainty. Given a fixed closed-loop save controller, we formulate the decision of when to initiate motion as a policy-conditional finite-horizon optimal stopping problem. Building on this formulation, we propose monotone optimal stopping (MOS), a structured release-timing method for dynamic robotic interception. The quadruped save policy is trained with reinforcement learning, while MOS determines when the policy should be activated from the evolving robot state and target belief. Rather than predicting a release time or relying on confidence alone, MOS learns the return advantage of acting now over waiting for one more observation. We derive a direct Bellman recursion for this act-versus-wait margin and impose monotonicity only with respect to physical urgency, reflecting the irreversible loss of interception opportunity as time elapses. This structure enables early activation for dynamically demanding saves while preserving closed-loop adaptation when later observations change the predicted target. Under a single-crossing condition, MOS admits a threshold release boundary with a bounded approximation error. Extensive simulation studies show that MOS improves the mean save rate from 67.7% to 74.4% over a parameter-matched learned gate and increases reversal saves from 52.1% to 66.5%. Real-robot experiments further demonstrate rapid interception and post-release direction correction under human shot-direction feints.
BibTeX
@misc{zhang2026anticipatory,
  author = {Hao E. Zhang and Ruize Geng and Yisen Li and Yaru Niu and Yikai Wang and Raihan Haque and Khalil Zbiss and Guanyang Luo and Hui-ping Wang and H. Eric Tseng and Ding Zhao},
  title = {{Anticipatory Robot Goalkeeping via Monotone Optimal Stopping}},
  year = {2026},
  eprint = {2609.23976},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2609.23976},
}
Figure