A facilitator, whether a teacher, a lab assistant or a parent, is supposed to step back while learners build something and step in only when they go seriously wrong. In hands-on spatial tasks such as assembling a molecule kit, a truss bridge or a wireframe polyhedron, keeping watch over every learner's build is tiring. A camera system that tells the facilitator "the strut at the top-left joint is off" could take over much of that watching. The catch is that such a system only helps if its alarms can be trusted. Every false alarm invites the facilitator to interrupt a learner who was actually right. Interrupting too early is exactly the over-intervention that facilitation research warns against, because it strips away learner initiative.
This proposal argues that, for builds made of thin parts, the main source of false alarms is a specific and fixable error in how 3D shape is recovered from photographs. Methods that turn multi-view images into 3D geometry by blending contributions along each camera ray tend to place surfaces slightly in front of or behind where they really are, and they smooth thin structures away. The research area called unbiased depth rendering has developed corrections for this error. We propose to use those corrections, together with a per-ray measure of how much error remains, to decide when the system should stay silent. We then test whether quieter, more accurate alerts change how facilitators intervene and how learners recover from their own mistakes.
Core Claim
The claim has two parts, and it is meant to be falsifiable. The first part is about perception. When a system checks a thin-strut assembly against its target design, using an unbiased surface reconstruction and gating alerts on the remaining rendering bias should cut false error alerts substantially at equal recall of real errors, compared with biased reconstruction, consumer depth sensors and image-only classifiers. The second part is about people. In a facilitated build session, that cut in false alarms should lead to fewer facilitator interventions per real error and more corrections that learners start themselves, without lowering what they learn. If an image-only classifier matches the 3D pipeline on alert accuracy, the 3D component is unnecessary, and the proposal reduces to a study of alert reliability and intervention behaviour. We say this up front because it is a live possibility.
Why the two directions meet here
Unbiased depth rendering addresses a known geometric error. Volume rendering is the technique of computing a pixel's colour by summing colour contributions along a ray, weighted by how much light reaches each point. When the geometry is stored as a signed distance function (a learned function whose zero level is the surface), those weights do not peak at the true surface. NeuS derived a weighting that removes this shift to first order. D-NeuS showed that the shift reappears on complex geometry and added a loss that penalises the distance value at the rendered surface point. RaNeuS tracks the gap between the zero crossing and the rendering peak, and relaxes smoothness on rays with high error to recover thin structures such as bicycle spokes. For point-based Gaussian splatting, PGSR replaces depth averaged across overlapping primitives, which is biased toward the camera, with exact ray–plane intersections and trains in about an hour. ND-SDF learns where external geometric priors disagree with the scene and treats that disagreement as a confidence map. These papers supply the method. All of them, however, judge success by average surface error over a whole scene. None asks whether a downstream yes/no decision, such as "is this part misplaced?", comes out right.
The facilitation literature supplies the motivation and the constraint. BrickSmart coached parents through children's block play using generated 3D targets and prompts. It reported a 49% gain in spatial-language scores against 18.23% for free play. Parents in that study also reported higher effort, partly because they had to check the physical build against the screen themselves; the system has no way to look back at what was built. Work on problem-based inquiry (Ge et al.) and the tentative, hedged prompting of the peer agent KSC-PaL both point the same way: guidance helps when it is selective and leaves the initiative with the learner. Together these define the requirement that the perception system has to meet. It must be selective, and above all it must not fire when the learner is right.
Research Program
We would build a fixed multi-camera rig over a desk, with eight to twelve cameras that are calibrated once. The task is a thin-strut construction kit, for example 3–5 mm rods with connector hubs, with known computer-aided design targets. At each build checkpoint the system reconstructs the current assembly. Because the target is known, the shape field starts from the target design rather than from scratch and only needs to be fine-tuned to the observed state. This is what we expect to bring reconstruction time from PGSR's roughly one hour per scene down to minutes; whether it does is an empirical question. Alerts arrive at checkpoints, not continuously, which matches how facilitators actually circulate around a room.
Error detection works by registering the reconstruction to the target and measuring, for each part, how far the observed surface deviates. The new ingredient is a reliability gate. For every ray that hits a part, we compute the residual bias $b(r) = |t^{*}_{w}(r) - t^{*}_{0}(r)|$, where $t^{*}_{w}$ is the depth at which the rendering weight peaks and $t^{*}_{0}$ is the depth of the field's zero crossing along ray $r$. This is the quantity D-NeuS and RaNeuS try to push to zero during training. Here we reuse it as a measure of local untrustworthiness. A part is flagged only if its deviation exceeds the build tolerance on rays whose residual bias is small. Where bias stays high, the system reports "uncertain" to the facilitator instead of "wrong". Alerts go to the facilitator, never directly to the learner. The facilitator sees the location of the suspected problem along with a hedged suggested prompt in the style of KSC-PaL, and decides whether to act.
Baselines
The naive reference point is no automated checking: the facilitator relies on their own eyes, as in BrickSmart's control condition. In the human study this condition defines baseline intervention rates and effort. On the perception benchmark its counterpart is a trivial detector that always predicts "correct", which fixes the floor for recall.
The first strong competitor is a consumer depth sensor of the kind built into tablets and RealSense cameras, fused into voxel occupancy and compared with the target using the same per-part deviation test. It is the cheapest serious alternative. It runs on the same rig with the same tolerances, and each sensor gets a sweep of its own deviation threshold. Beating it means higher precision at matched recall on thin parts. The file contains no reported thin-structure accuracy for such sensors, so this comparison has to be measured, not assumed.
The second is an image-only error classifier. This is a 2D network fine-tuned on multi-view crops of correct and perturbed builds, rendered from the design model and augmented with real captures, and it predicts for each part whether it is in place. It uses no 3D reconstruction at all and is the competitor most likely to embarrass the proposal. It gets the same training images and a compute budget at least equal to the reconstruction pipeline's per-checkpoint cost.
The strongest existing reconstruction methods are PGSR and RaNeuS, each run through our deviation test without the reliability gate. The best result on standard benchmarks reported in the file is RaNeuS at a DTU mean Chamfer distance of 0.51 mm (RaNeuS), against 0.61 mm for D-NeuS and Neuralangelo. PGSR reports the best Chamfer distance among Gaussian methods but gives no number in the file. We will first reproduce DTU numbers to confirm our implementations are faithful. For the task this proposal cares about, assembly error detection, no reported result was found on Lacuna for any method, so the bar is set by our own measurements.
Two ablations isolate the key ingredients. The first swaps PGSR's ray–plane depth back to alpha-blended depth while holding everything else fixed, which isolates the bias correction itself. The second removes the residual-bias gate from the unbiased pipeline, which isolates the gate. Beating the strongest baseline means a statistically significant reduction in false alerts per session at matched recall of real errors. As a target we set at least a halving; that figure is our own and is not drawn from any paper.
Why not just…?
Why not just use a tablet depth sensor? It is instant and cheap, and for chunky bricks of the kind BrickSmart uses it would probably be good enough. That is precisely why we choose thin struts, where sensor depth is likely to smooth over or miss parts. ND-SDF documents the analogous failure for learned monocular depth priors, which bake in errors at high-frequency structure. We have no measured sensor numbers for thin parts, though, so this alternative stays a real threat. If the sensor matches the gated pipeline on our benchmark, the right conclusion is that bias correction is unnecessary at this scale. We would then report that, and shift the study toward finer kits or drop the 3D component.
Why not just train a 2D classifier on images of right and wrong builds? This is the most serious alternative. A classifier can learn appearance cues directly and needs no geometry. Its weaknesses should show on unseen designs and on errors that differ from the training errors only in depth, such as a strut seated one hole deeper. A 3D deviation test generalises to those cases because it measures geometry instead of memorising what errors look like. The benchmark includes a held-out-design split specifically to settle this question. If the classifier wins there too, we abandon the 3D claim.
Why not just use vanilla Gaussian splatting, which trains fast and looks photorealistic? Its alpha-blended depth sits in empty space whenever a ray passes through semi-transparent primitives before reaching a surface (PGSR). On a thin strut, that shifted depth reads as a misplaced part, producing exactly the false alarm we are trying to remove. The first ablation tests this directly.
Why not just give learners direct step-by-step instructions and skip facilitation? That would make error checking simpler, but it gives up the learning benefits documented for inquiry-style guidance (Ge et al.; BrickSmart). Our study keeps facilitation fixed and changes only the reliability of the alerts.
Proposed Experiments
The perception benchmark comprises roughly 40 target designs, each captured in correct and controlled-error states. The errors are a missing strut, a wrong hub, a wrong angle, and depth-only seating errors, with ground-truth state recorded at assembly time. Metrics are per-part precision, recall and false alerts per checkpoint, reported across seen and held-out designs. Wall-clock latency per checkpoint is also reported, since minutes-scale alerts are a requirement. DTU Chamfer reproduction serves as a fidelity check, not as an outcome.
The human study is a between-subjects comparison of three conditions in facilitated build sessions with middle-school learners and trained facilitators: no alerts, alerts from the ungated or biased pipeline, and alerts from the gated unbiased pipeline. We will aim for a larger sample than the 24 pairs in BrickSmart and randomise at the facilitator level. Outcomes are facilitator interventions per true error, unnecessary interventions (coded from video against ground-truth build state), learner-initiated corrections, facilitator NASA-TLX workload, System Usability Scale scores, for which BrickSmart's 71.46 gives a reference, and a pre/post spatial reasoning and spatial-language test adapted from BrickSmart.
The decisive check links the two halves of the study. If the gated condition lowers unnecessary interventions and raises learner-initiated corrections relative to the ungated condition, while recall of real errors is equal, then the reliability of geometric verification, not just its presence, shapes how facilitation goes. If facilitators simply learn to ignore noisy alerts, so that intervention counts do not differ across conditions, the second half of the claim fails. That would be a useful negative result about how much perception accuracy facilitated instruction actually needs.
References
- NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstruction
- Recovering Fine Details for Neural Implicit Surface Reconstruction
- RaNeuS: Ray-adaptive Neural Surface Reconstruction
- PGSR: Planar-Based Gaussian Splatting for Efficient and High-Fidelity Surface Reconstruction
- ND-SDF: Learning Normal Deflection Fields for High-Fidelity Indoor Reconstruction
- BrickSmart: Leveraging Generative AI to Support Children's Spatial Language Learning in Family Block Play
- Exploring Engagement and Efficacy in Secondary English Education in China: A Problem-Based Social Constructivism Approach
- KSC-PaL: A Peer Learning Agent that Encourages Students to take the Initiative