TL;DR
When a human grabs a robot to correct its movement, the most important information might not be where they move it, but when they decided to intervene. This paper demonstrates that the timing of a human's correction is a rich signal that reveals their internal evaluation of the robot's performance and significantly speeds up the robot's ability to infer its intended goal.
What Problem Does This Paper Address?
In Interactive Robot Learning (IRL), corrections are typically treated as new demonstrations or preferences—the robot looks at the final corrected path to figure out what it did wrong. However, this approach ignores a crucial piece of data: the human’s decision to intervene in the first place.
Why did the person grab the robot at that exact second? Was the robot moving too slowly, heading toward the wrong object, or moving in an "unnatural" way? Most current models assume humans are "Boltzmann-rational," meaning they only intervene when the robot becomes mathematically sub-optimal. This paper argues that human intervention is driven by a much broader set of factors—like motion legibility and expectation alignment—and that the timing of these interventions can help a robot guess its goal much earlier than using spatial cues alone.
Methods
The authors propose a two-stage framework to model and utilize correction timing:
- Timing Prediction (The "WHEN" Model): They trained a Transformer-based model to predict the probability of a correction at any given timestep. Unlike simple models that only look at the current state, the Transformer accounts for the robot's entire trajectory history (making it non-Markovian). They fed the model seven time-series features, including:
- Expectation Alignment: How well the robot's velocity and position match a "natural" PID-generated path.
- Legibility: How clearly the robot's motion communicates its intent to a human.
- Optimality: The standard measure of efficiency.
- Directness and Velocity Consistency: Measures of how "smooth" and goal-oriented the motion appears.
- Goal Inference (The "COMBINED" Model): They developed a Bayesian framework to infer the goal distribution. They compared a WHERE model (using only the grasp position and initial velocity), a WHEN model (using only the timing of the intervention), and a COMBINED model that weights both signals.
Datasets
The researchers utilized a dataset from a previous large-scale user study:
- Participants: 120 people.
- Setup: A 7-DoF Kinova Gen3 robotic arm performing pick-and-place tasks (inserting shapes into color-coded holes).
- Data Scale: 7,435 interaction episodes and 3,585 physical correction trajectories.
- Ground Truth: To establish a "true" goal distribution, they used a sim-to-real approach, running 100,000 block-dropping simulations in PyBullet to see which release points actually resulted in successful task completion.
Key Insights and Ideas
- Human intervention is non-Markovian: People don't just react to the robot's current position; they react to the history of the motion. The Transformer model significantly outperformed baseline models that didn't consider trajectory history.
- Optimality isn't everything: While many RL papers rely solely on "optimality" to predict human behavior, this study found that features like Directness Alignment (moving straight to the goal) and Expectation Alignment were often more influential in triggering a human to intervene, especially mid-trajectory.
- Timing is a "Lead Indicator": The most striking "aha moment" is that timing information is most useful at the onset of a correction. It allows the robot to narrow down the intended goal before the human has even finished moving the arm.
Key Results
- Prediction Accuracy: The multi-feature Transformer model consistently outperformed the Boltzmann baseline in F1 score and Mean Absolute Error (MAE) for predicting correction timing.
- Early Goal Inference (RQ2): Using the Kullback–Leibler Divergence (KLD) to measure the distance between the inferred and true goal, the COMBINED model achieved significantly lower error than the spatial-only (WHERE) model for corrections occurring at 70%, 80%, and 90% of the trajectory.
- Diminishing Returns at the End (RQ3): Interestingly, timing information provided little to no benefit once the correction was finished. This is because, in simple tasks, the point where the human releases the robot is already a near-perfect indicator of the goal.
Limitations and Future Work
The authors acknowledge that their study focused on a relatively simple pick-and-place task where the mapping between a correction and a goal is fairly direct. In more complex, multi-constraint environments (e.g., "move the glass but don't spill it while avoiding the cat"), the spatial release point might be ambiguous, potentially making timing an even more critical signal.
Additionally, the current work is an offline analysis. The next major step is integrating this predictive timing model into real-time robot control, allowing a robot to proactively adjust its behavior the moment it senses it is about to be corrected, rather than waiting for the human to physically grab it.
Summary updated