In the world of robot manipulation, "long-horizon" tasks—like assembling a model airplane or setting a table—are the final frontier. While modern robots are getting better at simple "pick and place" actions, they often lose their way during multi-minute sequences. A key reason is the reward model: the internal "scorekeeper" that tells the robot how well it's doing.
Most current reward models are either too specific (trained for one task) or too vague (giving a single score for a complex video). Enter Grana, a new vision-language reward model from NVIDIA and university partners that uses a "divide and conquer" approach to track progress with unprecedented accuracy.
TL;DR
Grana is a vision-language reward model that decomposes long-horizon robot tasks into open-vocabulary subtasks to track continuous progress. By combining language-conditioned subtask selection with a novel "action-quality" supervision protocol, it achieves superior generalization and significantly boosts the success rates of downstream robot policies.
What Problem Does This Paper Address?
Tracking progress in long-horizon manipulation is notoriously difficult for two reasons:
- Generalization: Most models rely on fixed action vocabularies or task-specific classifiers. If you want the robot to do a new task, you often have to retrain the reward model from scratch.
- Imperfect Execution: Robots mess up. They drop screws, misalign parts, and have to try again. Simple "elapsed time" metrics fail here because time keeps moving forward even when progress has stalled or reversed.
The authors argue that a good reward model should be able to identify which subtask is currently being attempted and how much of that specific subtask is complete, all while being flexible enough to handle new tasks described in plain English.
Methods
Grana’s core innovation is its compositional approach to reward modeling.
1. Compositional Progress Prediction
Instead of predicting one global score, Grana uses a shared Vision-Language Model (VLM) backbone (Qwen3.5-4B) to:
- Select the Subtask: Given a list of language descriptions (e.g., "Place top wing," "Insert the screw"), it identifies which one is currently unfinished.
- Estimate Local Progress: It predicts a continuous score from 0 to 1 for that specific subtask.
- Compose Global Progress: Total progress is calculated by adding the credit for all previously completed subtasks to the current local progress.
2. Structured Progress Supervision
To train this, the authors moved away from asking humans to provide arbitrary numerical scores. Instead, they developed a protocol where annotators mark:
- Subtask Boundaries: When one step ends and the next begins.
- Action Quality (AQ): Whether the robot's current actions are Nominal (good), Suboptimal (struggling/misaligned), or Regressed (failure/setback).
These sparse labels are deterministically converted into a dense, "sawtooth" progress curve that drops to zero upon failure, providing a clear signal for the robot to learn recovery behaviors.
Datasets
- Pretraining: A massive corpus of 200 diverse manipulation tasks, mostly short-horizon (e.g., opening a toolbox, folding a shirt).
- Zero-Shot Evaluation: 50 held-out tasks to test generalization.
- Long-Horizon Benchmarks: Five complex tasks including Airplane Assembly (9 subtasks), Red Car Assembly (10 subtasks), and Pupgo Car Assembly (up to 18 subtasks).
- Public Benchmark: Evaluated on the RBM-EVAL-OOD trajectory ranking task.
Key Insights and Ideas
- Language is the Universal Interface: By conditioning on open-vocabulary descriptions, Grana can be applied to entirely new tasks without changing its architecture. You just give it a new list of strings.
- Short-Task Pretraining Transfers to Long-Horizon: One of the paper's most important findings is that learning "what progress looks like" on 200 simple tasks significantly improves the model's ability to adapt to complex, multi-minute tasks.
- Decomposition Beats Monoliths: Predicting progress for a 10-second subtask is a much easier learning problem than predicting it for a 5-minute assembly. The compositional logic handles the "long-term memory" so the neural network doesn't have to.
Key Results
- Zero-Shot Superiority: Grana achieved a 20.7% lower progress error (RMSE) than the strongest baseline (RoboMeter) on 50 unseen tasks without any fine-tuning.
- Data Efficiency: In long-horizon adaptation, Grana trained on just 15 trajectories outperformed baselines trained on 100 trajectories.
- Policy Boost: Using Grana's rewards for offline Reinforcement Learning led to a success rate of 65-67% on held-out tasks, compared to ~40-50% for baselines.
- Test-Time Planning: When used to guide a "Model Predictive Control" (MPC) planner on a tractor assembly task, success jumped from 2/10 to 9/10.
Limitations and Future Work
The authors note that their current composition logic assumes that once a subtask is finished, it stays finished. If a robot accidentally knocks over a previously placed wing while trying to insert a screw, the model might not automatically "un-credit" that progress unless the subtask selection logic is specifically designed to loop back.
Future work could involve more dynamic subtask graphs that allow for arbitrary transitions and even more complex "recovery" logic where the model can suggest which previous step needs to be redone.
Summary updated