The most accurate systems that find "where to grasp, sit on, or pour from" an unfamiliar 3D object now reason through a large multimodal language model step by step. That reasoning is too slow to run on a robot. The usual fix is distillation: run the slow model offline over many shapes, record its answers as synthetic labels, and train a small, fast network on them. The catch is that the offline pass is itself the main cost. Every shape sent to the large model is a multi-step query, and much of that spend is wasted on shapes that are functionally redundant. The hundredth mug handle teaches the student almost nothing the tenth did not.
We propose to decide which shapes get the expensive treatment using a cheap, hand-defined description of each object's functional geometry. This description records the orientation and height of flat surfaces relative to gravity, the slenderness and aspect ratio of parts, curvature, concavity, and likely interaction axes. Shapes and parts that share a functional-geometry signature share most of their affordance structure, so the teacher only needs to see a few from each group. Its labels can then be carried to the rest by geometric matching. The claim is that a student trained this way reaches the accuracy of full-budget distillation for a small fraction of teacher calls. The claim also predicts that the savings come specifically from function-aware grouping, not from generic diversity sampling.
Core Claim
Synthetic-label distillation for open-vocabulary 3D affordance grounding has a strongly diminishing return in teacher queries. Allocating those queries by functional-geometry strata flattens the return curve later and higher than either random allocation or allocation by learned shape embeddings. Concretely, a lightweight point-cloud student trained on teacher labels for roughly 10% of an unlabeled pool, chosen by functional-geometry stratification plus label propagation, should match a student trained on teacher labels for the whole pool. It should match within the noise of the PIADv2 unseen-affordance metric. Its errors should also be concentrated less on rare geometric configurations.
The proposal sits between two bodies of work that have not been joined. GREAT prompts InternVL through a four-step affordance chain of thought and fuses the resulting text into point-cloud features. The authors themselves flag the reasoning loop as too expensive for real-time robotics and point to distillation. This paper motivates our work and supplies the teacher. In text, TrueTeacher shows that brute-force distillation works: a 540B teacher labels 1.4 million examples for a 50x smaller student. It treats the labeling budget as a fixed price to pay. We treat that budget as the object of study. Beyond Quantity finds that synthetic code-agent trajectories saturate early with sample count and that diversity matters more per unit of compute. We test whether the same holds for 3D functional supervision, and whether functional diversity is the right axis rather than generic diversity. The functional-geometry descriptors come from an older line of work. That line decomposes shapes into support surfaces and vertical structures (Schlecht et al.), maps functional requirements to geometric precedents (Winston et al.), and turns spatial relations into surface-normal constraints (Kalita et al.). These works used geometry for recognition. We use it to direct annotation compute.
Research Program
We first build a part-level functional-geometry signature that runs on CPU in milliseconds per shape. Each point cloud is split into roughly planar, cylindrical, and residual regions. For each region we record its normal direction relative to the gravity axis, its height above the object's base, its extent and aspect ratio, its mean curvature and concavity, and its principal axis. We treat this as the per-part signature, and the object signature as the multiset of part signatures. Shapes in PIADv2 and 3D AffordanceNet are near-canonically upright, so we compute the signature in the gravity frame. Arbitrary rotation, which 3D AffordanceNet shows costs PointNet++ 5–10 mAP points, is handled separately in a stress test rather than assumed away.
We then cluster the unlabeled pool by object signature into strata. Under a teacher budget $B$, we allocate queries across strata in proportion to stratum size times within-stratum signature spread. Within each stratum we choose shapes by farthest-point selection in signature space. The teacher is the GREAT pipeline, which takes an interaction image and point cloud and returns dense per-point affordance maps for each affordance query. It is run only on the selected shapes. For every unselected shape, we match each of its parts to the nearest teacher-labeled part with the same affordance query in signature space. If the match distance is below a threshold, that part inherits the teacher's per-point scores, resampled by normalized position along the part's principal axis. If not, the part is left unlabeled for that query. The student is a text-conditioned PointNet++ in the style of Aff3DFunc. It is trained on direct teacher labels and propagated labels, with propagated labels down-weighted by match distance.
Cost is accounted in two ways: teacher forward passes, and measured teacher GPU-hours on fixed hardware. Descriptor computation and matching are charged to the method. We also log how often propagated labels disagree with a held-out teacher call, which is the proposal's analogue of the rejection rate that inflates synthetic pipelines elsewhere.
Baselines
The naive reference point is the student trained only on the human-labeled training data available at a small fraction, with no teacher at all. Next to it is the student trained with teacher labels on a uniformly random subset of size $B$. Random allocation is the honest "do nothing clever" policy, and every curve we report is plotted against it at matched $B$.
The ceiling is full-budget distillation, with the teacher run on every shape in the pool, as TrueTeacher does. We also report the teacher itself evaluated directly. GREAT reports 69.81% AUC on the PIADv2 unseen-affordance split, against 64.07% for LASO and 62.29% for IAG (GREAT). LASO, a direct grounding model with no language-model reasoning at test time, is therefore the strongest existing fast method with a published number. A student counts as useful if it clearly exceeds 64.07% AUC on that split. It counts as beating full-budget distillation on efficiency if it reaches the full-budget student's AUC within one standard deviation across three seeds, at no more than 15% of the teacher calls. We found no reported result on Lacuna for the cost or latency of GREAT's teacher pass, so we measure it ourselves.
The strongest competitor on the actual question, which is how to choose what to label, is coreset selection in a learned embedding. We embed each shape with a pretrained Point-MAE encoder, the same feature extractor Pair2Scene uses to ground support relations. We then pick $B$ shapes by k-center greedy selection, and propagate labels by nearest neighbour in that embedding. It uses the same pool, budget, student, training schedule, and hyperparameter search (an equal number of trials tuned on the same validation split). It differs only in replacing hand-built functional signatures with learned features.
Two ablations isolate the ingredients. The first uses functional stratification without propagation, which separates better selection from cheaper label coverage. The second uses propagation over random selection, which tells us whether the signature matters for transfer even when selection is naive.
Why not just…?
Why not just distill on everything, as TrueTeacher did? If the teacher pass were cheap, that would be the answer. It is the ceiling in our experiments, not a rival. The question is the shape of the curve below it. If full-budget and 10%-budget students are indistinguishable even under random selection, the problem is easier than we think and the contribution shrinks to a measured saturation point. The random-subset baseline reports this directly.
Why not just pick diverse shapes with a learned embedding? This is the real threat. Point-MAE features already encode geometry, and Pair2Scene shows they suffice to ground functional support relations without category labels. Our bet is that learned embeddings spend their diversity on appearance-level variation, such as overall silhouette and point density. Functional signatures spend it on what changes affordances, such as a horizontal surface at sitting height or a slender graspable axis. If the Point-MAE coreset matches us at every budget, we conclude that hand-built functional geometry adds nothing over learned features for this purpose, and we report that. The per-stratum error analysis below shows where any gap comes from.
Why not just use semi-supervised learning on the small human-labeled set? 3D AffordanceNet shows that virtual adversarial training learns usefully from as little as 1% labeled data. This method needs no teacher and is a credible cheap alternative for seen affordances. It cannot introduce affordance concepts that are absent from the labels, which is what the unseen-affordance split tests. We run it with the same student and the same 1% human labels, both alone and combined with our method, to see whether it is a substitute or a complement.
Why not just run the multimodal model at test time? GREAT's own authors describe the reasoning loop as an obstacle to real-time use. Scene-level analogues are slow too: Pair2Scene reports roughly 210 seconds per scene for Holodeck and 373 for LayoutVLM. We report teacher latency alongside student latency so the trade is explicit rather than asserted.
Proposed Experiments
The main experiment uses the PIADv2 training split as the unlabeled pool, with its human labels withheld except for a 1% reserve. Teacher budgets are 1%, 3%, 10%, 30%, and 100% of shapes. Every selection policy is plotted as student AUC, aIoU, and SIM on the seen-object, unseen-object, and unseen-affordance splits against teacher GPU-hours. We use three seeds per point. The decisive quantity is the budget at which each policy first comes within one standard deviation of the full-budget student. We predict that functional stratification with propagation gets there at around 10%, and that random and Point-MAE coreset selection need substantially more.
The second experiment tests where the savings come from. We bin test shapes by how rare their functional signature is in the pool and compare per-bin error across policies. If functional allocation helps mainly on rare geometric configurations, the gain is about coverage. This would be the 3D-affordance analogue of the rare-category jump Otsuka et al. saw when adding a few synthetic objects ("beam" from 4.3 to 25.9 AP). If instead the gain is uniform, it comes mostly from propagation-driven label volume.
The third experiment checks transfer and robustness. Students distilled on PIADv2 are fine-tuned with 1% labels on 3D AffordanceNet and compared against the reported PointNet++ full-shape result of 48.0 mAP. The same students are then evaluated under arbitrary rotation, to see whether gravity-frame signatures make selection brittle when the upright assumption fails.
Finally, we measure propagation fidelity by spending a small extra teacher budget on shapes that only received propagated labels, and reporting agreement with the teacher's direct output. High disagreement on concave or slender parts would show where the signature is too coarse. These are the same parts the file's sources flag as hard for existing networks.
References
- GREAT: Geometry-Intention Collaborative Inference for Open-Vocabulary 3D Object Affordance Grounding
- TrueTeacher: Learning Factual Consistency Evaluation with Large Language Models
- Beyond Quantity: Trajectory Diversity Scaling for Code Agents
- Learning models of object structure
- Learning Physical Descriptions From Functional Definitions, Examples, and Precedents
- Interpreting Prepositions Physically
- 3D AffordanceNet: A Benchmark for Visual Object Affordance Understanding
- Open-Vocabulary 3D Affordance Understanding via Functional Text Enhancement and Multilevel Representation Alignment
- Pair2Scene: Learning Local Object Relations for Procedural Scene Generation
- Pre-training with 3D Synthetic Data: Learning 3D Point Cloud Instance Segmentation from 3D Synthetic Scenes