TL;DR
GeneralVLA-2 introduces a powerful upgrade to hierarchical vision-language-action (VLA) robotics systems by fixing two critical bottlenecks: hallucinated 3D object geometry and unreliable experience reuse. By combining a mask-verified multi-view reconstruction branch (GeoFuse-MV3D) with a highly structured, Governed KnowledgeBank memory system, it enables safer, training-free, and more precise robot trajectory planning in both simulation and real-world environments.
What Problem Does This Paper Address?
Hierarchical VLA systems are highly attractive for robotics because they break down complex tasks into interpretable stages: perception, spatial reasoning (planning), and low-level control. However, their performance hinges heavily on the quality of the data fed to the mid-level planner. The authors identify two major pain points in existing architectures:
- Perception Hallucinations: Single-view or naive monocular 3D object reconstruction methods often hallucinate an object's pose or backside structure. For a robot arm, a few centimeters of error in geometry or pose estimation means the difference between a successful grasp and a catastrophic collision.
- Ungoverned Memory Retrieval: Previous long-term memory systems for agents rely almost entirely on simple semantic text similarity. If a robot faces a new task, it might retrieve a past experience that sounds similar but is geometrically irrelevant, or worse, blindly replay a strategy that previously failed.
Methods
GeneralVLA-2 addresses these challenges by restructuring the inputs that condition its 3D planning agent.
1. GeoFuse-MV3D Reconstruction Branch
When a robot can capture multiple short views around an object, it shouldn't have to guess its shape. GeoFuse-MV3D refines a baseline 3D Gaussian Splatting representation using two distinct geometry sources:
- Source A (Geometry-Prior Provider): Integrates external feed-forward 3D estimates (via VGGT) and applies an appearance calibration.
- Source B (Provider-Free Axis Compensation): Uses only the input masks and camera poses to calculate axis-wise scale and shift corrections.
Instead of aggressively deleting low-confidence points, the system applies a soft visual-hull refinement that gently shrinks low-support points inward. Crucially, it performs geometry-only fusion, blending the spatial $(x,y,z)$ coordinates while locking the original high-fidelity appearance fields (color, opacity, and spherical harmonics) to prevent visual drift.
2. Governed KnowledgeBank
The memory cache is upgraded into a rigorous database where every experience is stored as a structured record containing quality scores, confidence levels, lifecycle states (provisional, active, summary, archived), and conflict links.
- LLM-as-Verifier: After a task, a verifier evaluates the trajectory based on task completion, collision safety, and execution validity.
- Precision Retriever: Rather than just matching text, the retrieval function balances semantic relevance with success priors, recency, and explicit penalties for stale or conflicting records.
Datasets
The authors evaluated the individual modules and the unified system across several distinct benchmarks:
- GSO-30: A 30-object multi-view dataset derived from Google Scanned Objects, used to benchmark the 3D reconstruction pipeline.
- Terminal-Bench 2.0 & SWE-Bench Verified: Used to validate the governed memory module's performance on long-horizon software agent tasks independent of the physical robot stack.
- RLBench: 14 diverse simulation manipulation tasks (e.g., Close Box, Open Jar, Play Jenga) used for training-free robotic planning evaluation.
- Real-World Manipulation: Evaluated on a physical Agilex-2.0 Piper manipulator across four language-conditioned tasks (move spray bottle, open drawer, open jar, sort object).
Key Insights and Ideas
- Conservative Fusion Over Aggressive Deletion: In robotics, missing an object part due to over-filtering is far more dangerous than having a slightly noisy shape. Soft-shrinking low-support points preserves structural safety.
- Decoupling Geometry from Appearance: Fusing only spatial coordinates while keeping appearance attributes fixed prevents the color and opacity degradation that typically plagues multi-view fusion models.
- Failures as Constraints: A critical insight in the memory architecture is that failed trajectories shouldn't be discarded. Instead, they are explicitly tagged as negative constraints, ensuring the planner actively avoids repeating past mistakes.
Key Results
- Higher Fidelity 3D Models: On the GSO-30 benchmark, GeoFuse-MV3D outperformed the MV-SAM3D baseline across all metrics, reducing Chamfer Distance (CD) by 2.20% and LPIPS by 2.02%, while boosting PSNR by 2.36%.
- More Efficient Agent Planning: On Terminal-Bench and SWE-Bench Verified, the Governed KnowledgeBank improved success rates over the strong ReasoningBank baseline by 4.53% and 3.73% respectively, while consistently reducing the average number of steps (AS) and total token overhead.
- Robust Robot Execution: In simulation (RLBench), GeneralVLA-2 successfully generated trajectories for all 14 tasks training-free, outperforming baselines like VoxPoser and Hamster. In real-world robot trials, it achieved a high success rate of 83.33% on object sorting and 53.33% on opening jars, significantly outperforming zero-shot alternatives.
Limitations and Future Work
Despite its strengths, GeneralVLA-2 remains dependent on the reliability of its upstream inputs; severe mask leakage or camera calibration drift can negatively impact the soft visual-hull calculations. Furthermore, because the geometry fusion mechanism is intentionally highly conservative, the quantitative improvements in reconstruction metrics are steady but modest.
The authors note that the current real-world validation is restricted to static tabletop setups with short horizons. They point to long-horizon mobile manipulation, handling heavy occlusions, interacting with deformable objects, and implementing interactive human-in-the-loop recovery as exciting frontiers for future work.
Summary updated