Small vision-language models (VLMs, models that read images and answer in text) are poor at 3D reasoning. They judge an object's physical size by how many pixels it covers. They lose track of objects across camera views. Asked for a distance or an angle, they do little better than chance. The usual fix is to have a larger "teacher" model write step-by-step spatial reasoning and train the small "student" to copy it. The student then inherits the teacher's geometric mistakes, and the teachers are not good either: on FrontierSpatial the best evaluated model scores 45.20% against a 91.22% human baseline (FrontierSpatial). The other fix is reinforcement learning against a checkable reward, such as the overlap between a predicted and a true 3D box. That reward only scores the final answer, so it says nothing about which of dozens of reasoning steps went wrong.
This note proposes a way to get the benefits of both. Indoor 3D scans come with exact geometry: camera poses, object positions, and object sizes. Many intermediate statements in a spatial reasoning chain can therefore be checked exactly, for example "the chair is 2.1 m from the camera" or "the lamp is 40 cm wide." The student generates its own reasoning. At every such numeric claim, we ask the teacher what value it would have written there, and a geometry checker scores both values against the ground truth. The student is pulled toward the teacher only at claims where the teacher's value is measurably closer to the truth. Everywhere else, the teacher is ignored. This is where the student can overtake the teacher.
Core Claim
The teacher-dependent ceiling in spatial distillation can be broken by gating the teacher's supervision at the level of individual geometric claims, using a continuous error measure computed from scan geometry. The closest existing method gates at the level of a whole answer being right or wrong, and we argue that is the wrong granularity for geometry.
The nearest prior work is On-policy Distillation with Verifiable Reward (OPDVR). Its setting is on-policy distillation: the student writes a full solution and the teacher scores each token. OPDVR adds a gate based on whether the whole solution is correct. On correct solutions, the student is pulled toward the teacher only on tokens where the teacher is more confident. On incorrect solutions, the student is pushed down only where it was more confident than the teacher. On AIME24 this lifts a student to 36.9%, compared with 34.2% for ordinary on-policy distillation and 36.0% for the teacher itself. It is validated only on symbolic math with exact-match answers.
Porting OPDVR to 3D by thresholding box overlap into pass/fail is the obvious idea. It keeps two assumptions that fail for geometry.
First, it assumes answer-level correctness says something useful about each token. A spatial chain can reach a roughly right box through a wrong depth estimate that happens to cancel a wrong size estimate.
Second, it uses teacher confidence as a proxy for teacher quality. Numbers are written as text characters, so a teacher can be highly confident in the digit "3" of a distance that is off by a meter.
Our gate replaces confidence with the teacher's measured geometric error at that claim. It replaces answer-level correctness with claim-level error.
How the Gate Works
Training uses a fixed claim format. The student's reasoning must state spatial facts as structured spans, for example an object reference together with a depth, size, pairwise distance, or 3D center. A parser extracts each span $c$ from the student's rollout.
For each span we do three things. We take the student's value $c_S$. We greedily decode the teacher's value $c_T$, conditioned on exactly the same prefix. We compute errors $e(c_S)$ and $e(c_T)$ against ground truth from the scan: relative error for distances and sizes, and Euclidean error for centers. The teacher decode is cheap because spans are short and appear only a handful of times per chain.
The per-token learning signal combines two terms:
$$ A_t = \hat{A}_{\text{group}} + \lambda \, \mathbb{1}[t \in c,\; e(c_T) < e(c_S) - \delta] \, \big(\log \pi_T(y_t) - \log \pi_\theta(y_t)\big). $$
The symbols are as follows:
- $\hat{A}_{\text{group}}$ is the group-normalized advantage of the final-answer reward, as in Group Relative Policy Optimization (GRPO, which scores each rollout relative to other rollouts for the same prompt instead of training a separate value network).
- $\pi_T$ and $\pi_\theta$ are the teacher's and student's token probabilities.
- $\delta$ is a margin, so we do not chase noise in ground truth or parsing.
- $\lambda$ weights the distillation term.
At claims where the student already beats the teacher, the teacher term is zero. Optionally, a small positive bonus there reinforces the student's better value. Non-numeric reasoning tokens receive only the group advantage. We deliberately do not import OPDVR's confidence gate for those tokens, so that the contribution of the claim-level gate can be measured cleanly.
Research Program
Setting. The regime follows 3D-RFT:
- Student: Qwen3-VL-4B.
- Teacher: Qwen3-VL-32B-Thinking, the same teacher 3D-RFT used to write its warm-up reasoning data.
- Training scenes: ScanNet-derived video clips from the EmbodiedScan and ScanRefer annotations, which supply poses and oriented 3D boxes.
- Final-answer rewards: those of 3D-RFT, namely 3D box overlap for grounding, F1 for detection, and exact match for multiple-choice spatial questions.
Claim extraction and verification. We first regenerate a warm-up set in the claim format, so that claims are parseable. We then build the checker. The hardest engineering piece is mapping a textual object reference to an annotated scan object. Our plan is to require each claim to carry a 2D box in the referenced frame and match it to the projected ground-truth box. Claims that cannot be matched are dropped, and we report the drop rate.
Mechanistic analysis. Linear Mechanisms for Spatiotemporal Reasoning in Vision Language Models shows that VLMs encode near/far with the same linear direction they use for up/down in the image. We fit linear probes for depth and for image elevation on held-out scenes. We then measure whether the angle between the two directions grows more under claim-level gating than under the baselines. Our reasoning is that depth claims in scenes where depth and elevation disagree are exactly where a verifier should overrule both a conflating teacher and a conflating student.
We also evaluate the size-constancy items of SPHERE, where judging size by pixel area is the documented failure.
Baselines
Do nothing. The first reference point is the base Qwen3-VL-4B with no spatial training. A second naive reference is the teacher evaluated directly on every benchmark; this is the ceiling we claim to break. The file reports no teacher numbers on these benchmarks, so we measure them ourselves.
3D-RFT (strongest existing method). This is supervised warm-up on teacher reasoning followed by GRPO with geometric rewards, with the teacher discarded once RL begins. It reports a VSI-Bench average of 62.8 for 3D-RFT-4B, above VST-RL-3B at 57.7. It also reports a +6.5% gain in ScanRefer accuracy at 0.25 overlap over its baseline; the file gives no absolute value for that. It is the strongest competitor because it already uses verifiable geometric rewards at the same model scale.
We rerun it on our claim-format warm-up data with the same rollout count, steps, and hyperparameter search budget as our method, so that differences in data format cannot explain our gain. Beating it means a VSI-Bench average above both our reproduction and the reported 62.8. It also means higher ScanRefer accuracy at 0.25 overlap and higher ScanNet detection F1 at equal compute.
OPDVR ported to 3D. This is the obvious combination: OPDVR's confidence gate, with answer correctness defined as box overlap above 0.25 or an exact multiple-choice match. It runs on the same rollouts, teacher, and budget.
Ablations. Two ablations isolate our key ingredient. The first keeps claim-level teacher distillation but removes the error gate, so it distills at all claims. This tests whether gating matters at all. The second keeps the claim-level oracle but removes the teacher, rewarding the student directly for each claim's error. This tests whether the teacher contributes anything once intermediate claims are checkable.
For our method to count as working, it must beat both ablations. It must also exceed the teacher on the scan-derived evaluations, where the ceiling claim is actually testable.
Why not just…?
Why not just use 3D-RFT: warm up on teacher reasoning, then run GRPO? This is a real competitor. It also has the ceiling problem only in weakened form, because RL can move away from the teacher. Its reward, however, reaches only the final answer. The student gets no teacher help deciding which intermediate estimate to revise, and whatever systematic bias the warm-up instilled persists unless the final reward happens to expose it.
The decisive measurement is claim-level error on held-out scans, together with sample efficiency, meaning reward reached per rollout. If 3D-RFT matches us on both, the claim-level gate adds nothing, and we will say so.
Why not just port OPDVR with a thresholded overlap? Thresholding throws away most of the continuous signal. The confidence gate also trusts a confident teacher, which is precisely the failure mode for numbers written as characters. The ported-OPDVR baseline tests this directly.
We also compute a specific diagnostic: the fraction of claims where the confidence gate and the error gate disagree. If that fraction is small, the two methods are nearly equivalent, and the contribution reduces to engineering.
Why not just filter teacher traces by final-answer correctness and fine-tune on the survivors? SpatialTraceGen filtered teacher traces with an LLM judge. Raising the quality threshold improved trace scores by 17%, yet accuracy stayed flat at 74% on CLEVR-Humans. That suggests offline filtering does not remove the ceiling. SpatialTraceGen's judge was not a geometric oracle, though, so we add rejection-sampled supervised training on oracle-filtered teacher traces as an extra baseline rather than assume it fails.
Why not drop the teacher and reward verified claims directly? This is the most serious threat. If intermediate claims are checkable, a process reward needs no teacher. The teacher's possible value is in proposing which values to state, for example a reasonable depth prior on a hard frame, which pure exploration may take far longer to find.
The teacher-free ablation settles this. If it matches our method at equal compute, the honest conclusion is that claim-level verification, not distillation, is the contribution. The note would then be reframed around that finding.
Proposed Experiments and Decisive Checks
In-domain evaluation. This covers VSI-Bench, ScanRefer accuracy at 0.25 and 0.5 overlap, and ScanNet detection F1 in the 4-frame setting 3D-RFT reports. Because the VSI-Bench sensitivity diagnosis shows that video source and answer distributions confound rankings, we report the bias-normalized FN-VSI variant alongside raw scores. We also report STI-Bench's spatial-relation subset.
Transfer. We test on MV-RoboBench, whose multi-view robot scenes are outside the training distribution; its reported baselines are GPT-5 at 56.41% and Qwen2.5-VL-72B at 24.29%. We also test on the quantitative subset of FrontierSpatial, where evaluated models score below 17%.
Check 1: breaking the ceiling. On held-out ScanNet scenes, does the student's median claim error fall below the teacher's? This is measured on the same claim types, with teacher claims decoded from the teacher's own chains.
Check 2: the gate is responsible. The advantage should shrink sharply in the ungated ablation. It should shrink most on scenes where depth and image elevation are anticorrelated, where teacher bias should be largest.
Check 3: the teacher earns its place. Our method should reach a given reward level in fewer rollouts than the teacher-free ablation.
Check 4: the mechanism changes. The probe angle between depth and elevation directions, and SPHERE size-constancy accuracy, should improve more than under 3D-RFT.
Failing Check 1 would mean the idea does not break the ceiling. Failing Check 3 would mean the contribution is claim-level verification alone.
References
- FrontierSpatial: Holistic Evaluation of Spatial Reasoning in Multimodal Large Language Models
- On-policy Distillation with Verifiable Reward
- 3D-RFT: Reinforcement Fine-Tuning for Video-based 3D Scene Understanding
- Linear Mechanisms for Spatiotemporal Reasoning in Vision Language Models
- SPHERE: Unveiling Spatial Blind Spots in Vision-Language Models Through Hierarchical Evaluation
- SpatialTraceGen: High-Fidelity Traces for Efficient VLM Spatial Reasoning Distillation
- Hidden Sensitivity in Spatial Reasoning Evaluation: Diagnosis and Re-ranking with VSI-Bench