TL;DR
$\Psi$-0.6 is a unified 8.05-billion parameter transformer that integrates vision, language, and physical world modeling into a single reasoning process called "streams of thought." By using reinforcement learning to adaptively choose between visual, geometric (depth, flow), and textual representations, the model achieves state-of-the-art performance in spatial reasoning and counterfactual physical simulation.
What Problem Does This Paper Address?
Current AI systems for physical reasoning are often fragmented. We typically use separate, specialized models for different tasks: vision encoders for perception, vision-language models (VLMs) for description, and world models (WMs) for prediction. This separation makes it difficult to compose these capabilities—for example, using a predicted future state to answer a complex question—and prevents joint optimization of the entire reasoning chain.
The authors of $\Psi$-0.6 address this by asking: How can a single AI system learn which representations (e.g., raw pixels vs. geometric depth vs. optical flow) are most useful for solving a specific problem in the physical world?
Methods
The core of $\Psi$-0.6 is a unified multimodal architecture where images, camera poses, depth maps, optical flow, point tracks, and text are all serialized into a shared token interface.
Key technical components include:
- Unified Transformer: A decoder-only transformer (8B parameters) that processes all modalities as a single sequence.
- Streams of Thought: A reasoning process where the model interleaves different modalities. For instance, it might "think" by first estimating a camera pose, then generating a novel view, and finally answering a question based on that imagined view.
- Probabilistic Structure Integration (PSI): The model inherits a patch-based representation that allows fine-grained control over specific regions of an image and probabilistic predictions, enabling it to represent multiple plausible future outcomes.
- Adaptive Reinforcement Learning: The authors use Group Relative Policy Optimization (GRPO) to train the model to select the most effective "notation" (a sequence of intermediate representations) for a given task. Instead of a fixed reasoning pipeline, the model learns when it needs to "see" depth or "imagine" flow to improve its final answer.
Datasets
The model underwent a multi-stage training process using a diverse array of data:
- Pre-training: Large-scale video datasets covering natural scenes, human activity, and robotics (e.g., Ego4D, DROID, Bridge).
- Mid-training: Text-only data (FineWeb-Edu, DCLM) and vision-language alignment data (COCO, PixMo-Cap).
- Evaluation Benchmarks:
- MindCube: A challenging benchmark for multi-view spatial reasoning.
- FlowBench (Sim & Real): A benchmark for counterfactual physical reasoning (e.g., "What happens if I move this box?").
- RL Generalization: Evaluated on unseen benchmarks including HanDyVQA, TGIF-QA, STAR, and EgoPlan-Bench2.
Key Insights and Ideas
- Unification as a Reasoning Engine: By treating perception and imagination as token-prediction tasks within the same model, $\Psi$-0.6 turns "imagining" into a first-class reasoning operation. The model that interprets a scene is the same one that predicts its future.
- Geometry as a "Mental Sketchpad": The paper demonstrates that explicit geometric intermediates (like depth and flow) act as powerful abstractions. They allow the model to decompose complex physical problems into simpler steps, much like a human might mentally rotate an object to see if it fits.
- Learnable Meta-Reasoning: Perhaps the most significant insight is that the choice of how to reason can be optimized. Through RL, the model learns that certain questions are best answered by looking at depth, while others require simulating motion.
Key Results
- Spatial Reasoning: On MindCube, $\Psi$-0.6 outperforms leading open-weight generalist models like DeepSeek-VL2 and LLaVA-OneVision. Using "compositional prompting" (inferring poses to select the right view) boosted zero-shot accuracy from 51.3% to 57.7%.
- Counterfactual Imagination: On FlowBench, the model's world-modeling capabilities surpassed dedicated expert models. It achieved 56.4% flow prediction accuracy, significantly higher than image-editing baselines (16.4%).
- RL Gains: The RL-trained policy for selecting streams of thought showed a 9.0% mean relative accuracy gain across four completely unseen benchmarks, proving that the learned reasoning strategies generalize well beyond the training data.
Limitations and Future Work
- Scale: The authors note that $\Psi$-0.6 was developed at a much smaller scale (8B parameters) than frontier multimodal systems. They expect significant performance gains from scaling up data and compute.
- From Reasoning to Action: While the model can reason about physical actions (e.g., predicting flow), it does not yet directly control a robot. A key next step is integrating robot joint commands and end-effector poses as token modalities for true embodied AI.
- Autodiscovery of Representations: Currently, the model chooses from a predefined set of human-understandable representations (depth, flow, etc.). Future work could involve letting the model discover its own latent, compact representations that are optimized for physical consistency.
Summary updated