Enhancing Goal Inference via Correction Timing

Anjiabei Wang, Shuangge Wang, Tesca Fitzgerald

TL;DR

When a human grabs a robot to correct its movement, the most important information might not be where they move it, but when they decided to intervene. This paper demonstrates that the timing of a human's correction is a rich signal that reveals their internal evaluation of the robot's performance and significantly speeds up the robot's ability to infer its intended goal.

What Problem Does This Paper Address?

In Interactive Robot Learning (IRL), corrections are typically treated as new demonstrations or preferences—the robot looks at the final corrected path to figure out what it did wrong. However, this approach ignores a crucial piece of data: the human’s decision to intervene in the first place.

Why did the person grab the robot at that exact second? Was the robot moving too slowly, heading toward the wrong object, or moving in an "unnatural" way? Most current models assume humans are "Boltzmann-rational," meaning they only intervene when the robot becomes mathematically sub-optimal. This paper argues that human intervention is driven by a much broader set of factors—like motion legibility and expectation alignment—and that the timing of these interventions can help a robot guess its goal much earlier than using spatial cues alone.

Methods

The authors propose a two-stage framework to model and utilize correction timing:

  1. Timing Prediction (The "WHEN" Model): They trained a Transformer-based model to predict the probability of a correction at any given timestep. Unlike simple models that only look at the current state, the Transformer accounts for the robot's entire trajectory history (making it non-Markovian). They fed the model seven time-series features, including:
  2. Expectation Alignment: How well the robot's velocity and position match a "natural" PID-generated path.
  3. Legibility: How clearly the robot's motion communicates its intent to a human.
  4. Optimality: The standard measure of efficiency.
  5. Directness and Velocity Consistency: Measures of how "smooth" and goal-oriented the motion appears.
  6. Goal Inference (The "COMBINED" Model): They developed a Bayesian framework to infer the goal distribution. They compared a WHERE model (using only the grasp position and initial velocity), a WHEN model (using only the timing of the intervention), and a COMBINED model that weights both signals.

Datasets

The researchers utilized a dataset from a previous large-scale user study:

  • Participants: 120 people.
  • Setup: A 7-DoF Kinova Gen3 robotic arm performing pick-and-place tasks (inserting shapes into color-coded holes).
  • Data Scale: 7,435 interaction episodes and 3,585 physical correction trajectories.
  • Ground Truth: To establish a "true" goal distribution, they used a sim-to-real approach, running 100,000 block-dropping simulations in PyBullet to see which release points actually resulted in successful task completion.

Key Insights and Ideas

  • Human intervention is non-Markovian: People don't just react to the robot's current position; they react to the history of the motion. The Transformer model significantly outperformed baseline models that didn't consider trajectory history.
  • Optimality isn't everything: While many RL papers rely solely on "optimality" to predict human behavior, this study found that features like Directness Alignment (moving straight to the goal) and Expectation Alignment were often more influential in triggering a human to intervene, especially mid-trajectory.
  • Timing is a "Lead Indicator": The most striking "aha moment" is that timing information is most useful at the onset of a correction. It allows the robot to narrow down the intended goal before the human has even finished moving the arm.

Key Results

  • Prediction Accuracy: The multi-feature Transformer model consistently outperformed the Boltzmann baseline in F1 score and Mean Absolute Error (MAE) for predicting correction timing.
  • Early Goal Inference (RQ2): Using the Kullback–Leibler Divergence (KLD) to measure the distance between the inferred and true goal, the COMBINED model achieved significantly lower error than the spatial-only (WHERE) model for corrections occurring at 70%, 80%, and 90% of the trajectory.
  • Diminishing Returns at the End (RQ3): Interestingly, timing information provided little to no benefit once the correction was finished. This is because, in simple tasks, the point where the human releases the robot is already a near-perfect indicator of the goal.

Limitations and Future Work

The authors acknowledge that their study focused on a relatively simple pick-and-place task where the mapping between a correction and a goal is fairly direct. In more complex, multi-constraint environments (e.g., "move the glass but don't spill it while avoiding the cat"), the spatial release point might be ambiguous, potentially making timing an even more critical signal.

Additionally, the current work is an offline analysis. The next major step is integrating this predictive timing model into real-time robot control, allowing a robot to proactively adjust its behavior the moment it senses it is about to be corrected, rather than waiting for the human to physically grab it.

Summary updated

Abstract

Corrections offer a natural modality for people to provide feedback to a robot, by (i) intervening in the robot's behavior when they believe the robot is failing (or will fail) the task objectives and (ii) modifying the robot's behavior to successfully fulfill the task. Each correction offers information on what the robot should and should not do, where the corrected behavior is more aligned with task objectives than the original behavior. Most prior work on learning from corrections involves interpreting a correction as a new demonstration (consisting of the modified robot behavior), or a preference (for the modified trajectory compared to the robot's original behavior). However, this overlooks one essential element of the correction feedback, which is the human's decision to intervene in the robot's behavior in the first place. This decision can be influenced by multiple factors including the robot's task progress, alignment with human expectations, dynamics, motion legibility, and optimality. In this work, we investigate whether the timing of this decision can offer a useful signal for inferring these task-relevant influences. In particular, we investigate three potential applications for this learning signal: (1) identifying features of a robot's motion that may prompt people to correct it, (2) quickly inferring the final goal of a human's correction based on the timing and initial direction of their correction motion, and (3) learning more precise constraints for task objectives. Our results indicate that correction timing results in improved learning for the first two of these applications. Overall, our work provides new insights on the value of correction timing as a signal for robot learning.
BibTeX
@misc{wang2026enhancing,
  author = {Anjiabei Wang and Shuangge Wang and Tesca Fitzgerald},
  title = {{Enhancing Goal Inference via Correction Timing}},
  year = {2026},
  eprint = {2602.18603},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2602.18603},
}
Figure