FLORA: An Open Model Suite for Scaling Robotics Data Engines

Sicong Leng, Yunze Man, Zhiqi Li, Jing Wang, Yuhe Zhang, Ryan Punamiya, Siddharth Patki, Kaiyuan Zheng, Zi-ang Cao, Zhiding Yu, Sifei Liu, Danfei Xu, Ben Levin, Jan Kautz, Yuke Zhu, Johan Bjorck, Linxi "Jim" FanarXiv 2026 · arXiv

TL;DR

FLORA is an open-source, industrial-grade data engine from NVIDIA designed to scale robotics foundation models by automating the generation of timestamped captions and multi-granular rewards. It achieves performance competitive with frontier models like GPT-6 Astra at a staggering 125x lower cost, enabling the labeling of 200,000 hours of video data per day.

What Problem Does This Paper Address?

The development of robotics foundation models, such as NVIDIA’s GR00T, is bottlenecked by the need for high-quality, large-scale data labels. Specifically, researchers need:

  1. Timestamped captions to filter and balance massive pretraining datasets.
  2. Process rewards (dense, step-by-step feedback) for reinforcement learning (RL) post-training.
  3. Outcome rewards (sparse, success/failure labels) to monitor real-world deployments.

Existing solutions are either too slow (human labeling), too narrow (specialized models), or prohibitively expensive at scale (frontier LLM APIs). FLORA fills this gap by providing a unified, open-weight suite that handles all these tasks with industrial reliability and efficiency.

Methods

FLORA is built on the Qwen3.5 family of vision-language models (VLMs), utilizing a 9B parameter model for segmentation and a 27B model for captioning and rewards. The system employs two primary pipelines:

  • Captioning Pipeline: A two-stage process where the video is first segmented into temporal intervals using sliding windows and a stitching algorithm. These segments are then captioned using a "contextual action" approach that looks at the visual evidence immediately before, during, and after the target event to resolve object identity and intent.
  • Reward Modeling: A single model trained to produce three types of rewards: sparse outcome scores (1–5 scale), dense process rewards (labeling every 0.5s as normal, suboptimal, or failure), and flexible rewards based on user-defined rubrics provided at test-time.
The FLORA pipeline for timestamped captions and rewards.

The models were trained using Supervised Fine-Tuning (SFT) followed by Reinforcement Learning via Group Relative Policy Optimization (GRPO), using verifiable rewards for segmentation and reference-grounded rewards for captioning.

Datasets

The authors evaluated FLORA on several established benchmarks, including WGO-Bench (captioning), RoboRewardBench (outcome rewards), and RobometerBench (trajectory ranking).

Crucially, they introduced three new benchmarks to address limitations in existing ones:

  • DenseCaptionBench: 300 complete episodes across human, parallel-gripper, and dexterous-hand embodiments to test multi-view and high-dexterity scenarios.
  • DenseRewardBench: 40 bimanual episodes with 4,861 human-labeled time steps to evaluate failure detection.
  • RubricBench: 40 teleoperated episodes graded against human-created rubrics to test flexible reward modeling.

Key Insights and Ideas

  • The Efficiency Frontier: FLORA demonstrates that specialized, smaller VLMs can match the quality of massive frontier models for specific domains like robotics. By optimizing for the task, FLORA achieves a massive cost reduction without sacrificing performance.
  • Temporal Context is King: For action captioning, the "before" and "after" frames are not just extra data; they are essential for understanding what the robot is trying to do and why, significantly reducing object and intent confusion.
  • Failure Detection Advantage: One of FLORA's most significant "aha" moments is its superior failure recall. While frontier models often struggle to identify the exact moment a robot fails, FLORA recalls 53.3% of failure steps, compared to less than 12% for other baselines.
  • Industrial Scalability: The engine is "battle-tested," currently labeling 200,000 hours of data daily across 512 GB300 GPUs, proving it is a ready-to-use tool for large-scale robotics research.

Key Results

FLORA achieves state-of-the-art (SOTA) results across multiple benchmarks, proving it is both a high-performance and high-efficiency engine.

  • Cost/Quality: FLORA achieves 57.1% annotation quality at a cost of $0.72 per video hour**, compared to GPT-6 Astra’s 53.7% quality at **$88.49 per hour—a ~125x cost advantage.
  • Captioning: On WGO-Bench, FLORA leads in segmentation F1 (50.3%) and is highly competitive in joint semantic F1.
  • Rewards: FLORA-Rewarder achieved the lowest Mean Absolute Error (40.0) on RoboRewardBench, a 27.8% reduction in error relative to GPT-6 Astra.
Annotation quality versus cost comparison.

Limitations and Future Work

Despite its strengths, the authors note several areas for improvement:

  • Over-segmentation: The model can sometimes fragment long, coarse actions into too many short segments.
  • False Positives: The high recall for failures comes at a cost; FLORA incorrectly labels about 11% of normal steps as failures.
  • Complex Coordination: While it excels at human coordination, it can still confuse action phases or objects in highly complex dexterous manipulation tasks.

By open-sourcing FLORA, NVIDIA aims to empower the broader research community to scale their own robotics datasets and explore new frontiers like human-to-robot transfer and physical auto-research.

Summary updated

Abstract

The life-cycle of robotics foundation models crucially depends on data labels. Outcome reward labels are needed for monitoring real-world deployments. Process reward labels are needed during RL post-training. Timestamped captions are needed for filtering and balancing pretraining data. We present Flora, an industrial-quality data engine used for training GR00T models, that simultaneously covers these capabilities. Flora achieves SOTA scores on the captioning benchmark WGO-Bench and reward benchmarks RoboReward and Robometer. On internal benchmarks it is competitive with GPT-6 Astra while beating other frontier models. The model is substantially cheaper than API-based alternatives, roughly 125 times cheaper than GPT-6 Astra. Florais battle tested at scale, labeling 200,000 hours of data per day across 512 GB300s. By open-sourcing our data engine, we enable researchers to scale custom robotics datasets and explore emerging applications such as physical auto-research and human-to-robot transfer.
BibTeX
@misc{leng2026flora,
  author = {Sicong Leng and Yunze Man and Zhiqi Li and Jing Wang and Yuhe Zhang and Ryan Punamiya and Siddharth Patki and Kaiyuan Zheng and Zi-ang Cao and Zhiding Yu and Sifei Liu and Danfei Xu and Ben Levin and Jan Kautz and Yuke Zhu and Johan Bjorck and Linxi "Jim" Fan},
  title = {{FLORA: An Open Model Suite for Scaling Robotics Data Engines}},
  year = {2026},
  eprint = {2610.31779},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2610.31779},
}
Figure