Learned Termination Actions for Autonomous Task Completion

This direction is merged into another direction: premature task termination. Its last version stays readable and citable here.

Version 1 of 19 papers17 conceptsfrozen 2026-10-04

In classical reinforcement learning (RL), an episode typically ends when the environment detects that a goal condition has been met. A robot reaches a coordinate, a game score hits a threshold, or a timer runs out, and the simulation resets. While convenient for training, this "environment-triggered" termination acts as a crutch. In the real world, there is no external signal that tells a robot it has successfully grasped an object or reached the correct room.

To bridge this gap, researchers have moved toward learned termination actions. By treating "I am done" as an explicit action within the agent’s own control, models must develop an internal representation of success.

The Problem of Accidental Success

When the environment terminates an episode based solely on proximity to a goal, it becomes difficult to distinguish between an agent that knows where it is going and one that simply stumbled into the target.

In On Evaluation of Embodied Navigation Agents, Angel X Chang, Saurabh Gupta, and their colleagues argue that true navigation requires the agent to explicitly signal completion. They introduced the Success weighted by Path Length (SPL) metric, which penalizes agents that wander aimlessly before hitting the goal. By requiring a termination action, we ensure the agent possesses a "verifiable internal understanding" of its progress. If an agent reaches the goal but fails to signal, it hasn't truly succeeded; it has merely arrived.

Replacing Brittle Heuristics with Learned Confidence

In robotic manipulation, determining success is notoriously difficult. A robot might have its fingers around a block, but is the grip secure? Traditional systems use hard-coded heuristics—like checking if a gripper is closed beyond a certain point—to decide when to stop.

In QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulation, Alex Irpan, Dmitry Kalashnikov, and the Google Brain team demonstrated that incorporating a discrete termination action into a continuous action space allows the agent to adapt its confidence to the visual state. Instead of relying on a sensor threshold, the agent learns to "stop" only when its visual feedback confirms a successful grasp. This leads to emergent behaviors like regrasping and probing, where the agent refuses to terminate until it is certain of success.

Preventing the Undo Problem

A unique challenge arises in complex reasoning tasks: an agent might reach a successful state but then immediately move out of it because it doesn't know it should stop. This is particularly prevalent in natural language tasks where the goal is a specific configuration of objects.

Working on the lilGym benchmark, Anne Wu and Yoav Artzi observed that agents often struggle to identify the optimal moment to terminate after satisfying a linguistic condition. To solve this, they employed termination action forcing—masking all other actions once a goal state is reached during training. This forces the agent to observe the high reward associated with the "stop" action, preventing it from inadvertently undoing its own hard work through continued exploration.

Termination as a Modular Handoff

Learned termination is also the glue that holds hierarchical systems together. When a complex task is broken down into subtasks, the system needs a way to transition between them.

In Modular Multitask Reinforcement Learning with Policy Sketches, Jacob Andreas and Sergey Levine used specialized termination actions to signal the completion of subpolicies. When a "pick up" module triggers its termination action, control is automatically passed to the "carry" module. This allows for a clean separation of concerns: each subpolicy only needs to know how to achieve its specific end-state and signal that it is done, without needing a global view of the entire multi-step mission.

Real-World Reliability

The transition from simulation to reality often breaks agents that rely on environment "cheats." Jonas Kulhanek and his team found that for Visual Navigation in Real-World Indoor Environments, a dedicated termination signal was essential. By requiring the robot to trigger this signal within a specific radius of the target, they prevented the common failure mode where a physical TurtleBot would reach a destination but continue to wander until it crashed or timed out.

By moving the "stop" signal from the environment's code into the agent's brain, these researchers have created agents that are more autonomous, more interpretable, and significantly more capable of handling the ambiguity of the real world.

Papers in this version

History & citationVersion 1 · 2026-10-04

Merged into another direction · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_learned_termination_actions_for_autonomo_11841b24,
  title = {Learned Termination Actions for Autonomous Task Completion},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/learned-termination-actions-for-autonomous-task-completion/rdr_11841b24f4cb49c7b87dc021bd9bbd2c},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-22},
}

This exact version (version 1, frozen text):

@misc{lacuna_learned_termination_actions_for_autonomo_11841b24_v1,
  title = {Learned Termination Actions for Autonomous Task Completion},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/learned-termination-actions-for-autonomous-task-completion/rdr_11841b24f4cb49c7b87dc021bd9bbd2c/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-22},
}