Single-image 3D reconstruction models now turn a photo into a 3D object in seconds. They do it by predicting a tri-plane: three flat, axis-aligned feature images whose values are looked up and combined to describe any 3D point. They are also large. LRM uses a 500M-parameter transformer and needs about 5 seconds on an A100. The natural next step is knowledge distillation, where a small "student" network is trained to imitate a large "teacher." The obvious recipe copies the teacher's three feature planes directly, which is what Head3D does for one plane of a face generator. We argue this recipe asks the student to copy the wrong thing. A tri-plane is not unique: many different sets of planes decode to exactly the same 3D object. Forcing a smaller student to reproduce the teacher's particular planes wastes its limited capacity on matching details that never affect the output.
The proposal is to distill in field space instead. The student is trained to match what the teacher's planes produce at sampled 3D points: density, color, and the decoded per-point features. On top of the per-point match, a relational term asks the student to reproduce which points the teacher treats as similar to which others. The central prediction is specific. Plane-matching distillation should degrade faster than field-space distillation as the student's planes get smaller, because a smaller student has less spare capacity to spend on copying the teacher's arbitrary choice of planes. The relational term should help most in the regions the input photo never showed, where these models produce blurry averages.
Core Claim
In the standard tri-plane pipeline, a 3D point $p$ is projected onto the three planes. A feature vector is read from each plane, and the three vectors are summed (or concatenated) into $f_p$, which a small network decodes into density $\sigma$ and color $c$ (Convolutional Occupancy Networks introduced this planar factorization).
The representation has an exact redundancy, which we will call a gauge freedom, borrowing the physics term for choices that change the description but not the thing described. Under summation, adding any constant vector $a$ to the XY plane and subtracting $a$ from the XZ plane leaves every $f_p$ unchanged. More generally, any invertible linear map applied to the features can be undone by the first layer of the decoding network. A plane-space loss such as $\lVert P^S - P^T \rVert^2$, comparing student planes $P^S$ to teacher planes $P^T$, penalizes a student that renders identically to the teacher but chose a different description. When the student's planes have lower resolution or fewer channels than the teacher's, an exact copy is impossible anyway. The loss then pushes toward a compromise copy of the planes rather than toward the best achievable 3D object.
We instead minimize a field-space objective over a set $\mathcal{P}$ of sampled 3D points:
$$ \mathcal{L} = \sum_{p\in\mathcal{P}} \Big(|\sigma^S(p)-\sigma^T(p)| + \lVert c^S(p)-c^T(p)\rVert_1\Big) + \lambda \sum_{(p,q)} \ell_\delta\!\big(\hat d^S(p,q)-\hat d^T(p,q)\big) $$
The symbols are defined as follows. $\sigma$ and $c$ are density and color from the student (S) or teacher (T). $\hat d(p,q)$ is the distance between the decoded features $f_p$ and $f_q$, divided by the mean pairwise distance in the sampled set. $\ell_\delta$ is the Huber loss, and $\lambda$ sets the weight of the relational term. Both terms are unchanged by plane gauge transformations. The relational term, in addition, does not care whether student and teacher features share a dimension or a coordinate system.
The relational term is the distance-wise loss of Relational Knowledge Distillation (RKD). RKD transferred the geometry of the teacher's feature space and let a ResNet18 student exceed its ResNet50 teacher on Cars 196. Here we move it from images in a batch to points in a 3D field. We use only the pairwise (distance) version and sample a few thousand pairs per object, because RKD's angle version scales cubically with the number of points.
Research Program
Gauge diagnostic. The first piece of work establishes that the problem is real rather than theoretical. We take a pretrained tri-plane teacher and apply random gauge transformations to its planes, folding each compensating linear map into the decoder. We confirm that the renders stay bit-for-bit equivalent while plane-space distance becomes large. Then, across checkpoints of students trained with plane-space distillation, we measure how weakly the plane-space loss tracks actual rendering and geometry error.
Teacher and student. For the main study, the teacher is the tri-plane reconstruction model used as the InstantMesh backbone in Freeplane. Freeplane reports InstantMesh's numbers on Google Scanned Objects, so the teacher has a published reference point. The student is a transformer of the same family with a smaller image encoder and decoder. Its tri-plane is reduced in both spatial resolution and channel count. We train a ladder of students with progressively smaller planes so that the compression ratio becomes an experimental axis rather than a single point.
Training data. Training images come from Objaverse renders, as in LRM. Because the field-space loss needs only teacher queries, we can also distill on input images that have no 3D ground truth.
Choosing the 3D points. Point sampling is a deliberate design choice. One third of the points lie near the teacher's surface (high density). One third lie in regions not visible from the input camera. One third are uniform in the volume.
The unseen-region points matter because the file documents two related failures. Feed-forward tri-plane models regress toward blurry "mean" appearance in unseen regions (LRM, TGS). Separately, points that fall on the same projection line share plane cells, so their features interfere with each other (Vertical Feature Collapse in Spatial Projections). That interference worsens as plane resolution drops. Our hypothesis is that the relational term tells the student which unseen points belong to the same part as visible ones. That grouping information is precisely what a per-point regression target averages away.
Relation to prior work. Head3D supplies the plane-space distillation recipe that we treat as the main competitor. Head3D distills only the frontal plane of a face-only EG3D generator into a full-head generator. Its teacher and student share architecture and plane layout, so the gauge problem is mild there. Our setting differs on both counts: the student is smaller than the teacher, and the student must imitate all three planes. Fused-Planes motivates compression by showing that tri-planes are highly redundant: shared base planes cut per-object storage 1875×. However, it trains per-category planes from scratch and does not distill a feed-forward model. ControLRM reuses frozen LRM weights for control but never compresses them. The 2D-to-3D distillation work, PartDistill, OV3D, PartSAM and CF3, distills 2D image models into point clouds, sparse voxels, a frozen tri-plane branch, or Gaussians. None of these uses a 3D tri-plane teacher distilled into a smaller tri-plane student, which is our case.
Baselines
The naive reference point is the same student architecture trained from scratch on Objaverse ground-truth renders, with the teacher's original rendering losses (pixel error plus LPIPS, a learned perceptual similarity metric). The data, number of optimization steps, and hyperparameter search budget match our method. Any distillation method must beat this student to justify having a teacher at all. The teacher itself is the ceiling. Freeplane reports the InstantMesh teacher on GSO at Chamfer Distance 16.34, Volume IoU 0.429 and Normal Consistency 68.20 (Freeplane). We will re-measure the teacher under our own protocol rather than mix protocols.
The strongest competitor is plane-space feature distillation in the style of Head3D. The student regresses all three teacher planes through a learned 1×1 convolution and resampling adapter, which bridges the channel and resolution mismatch, alongside the ground-truth rendering loss. It is the strongest competitor because it is the only published tri-plane-to-tri-plane distillation method. Head3D also shows that this kind of distillation decisively beats direct fine-tuning (FID 11.34 versus 30.46). Head3D's numbers come from a face GAN, so no reported result transfers to GSO reconstruction. We will therefore run this baseline ourselves with the same student, data, points budget, and an equal hyperparameter sweep over its loss weight.
The second strong competitor is render-space distillation. The student is supervised with teacher-rendered images from many viewpoints, in addition to ground truth. This loss is also unaffected by plane gauge, so it isolates whether supervising 3D points adds anything beyond supervising 2D views.
For geometry context we also report TGS. TGS reports GSO PSNR 23.15 and Chamfer Distance 21.04 at 0.14 seconds per reconstruction. It is a fast tri-plane-plus-Gaussian reconstructor, so it is the natural speed–quality reference for a compact student.
Two ablations isolate the key ingredients: field-space per-point matching without the relational term, and the relational term with uniform rather than unseen-region-weighted points.
Beating the field means three things at a fixed student size. Our student must outperform plane-space distillation on GSO Chamfer Distance and LPIPS. Its margin over plane-space distillation must grow as student planes shrink. It must close a clearly larger fraction of the scratch-to-teacher gap than render-space distillation, especially on metrics restricted to unseen regions.
Why not just…?
Why not just train the small model from scratch on the same data? The teacher has seen the same data, so on paper it adds nothing. In practice the teacher provides dense, consistent 3D targets, including inside the volume and on the unseen side, where ground-truth renders give only indirect signal through rays. The scratch baseline measures this gap directly. If the gap is negligible, distillation of any kind is unnecessary at this scale, and that is a reportable result.
Why not just distill from rendered images? This is the most serious threat. Render-space distillation is already immune to the gauge problem and is simple to implement. Our bet is that images integrate density along rays, so they underconstrain interior geometry and the unseen side. This is the same surface-only limitation PartSAM documents for 2D lifting. The per-point-only ablation will tell us whether 3D supervision beats 2D supervision at equal teacher-query cost. If it does not, we will report that the gauge insight argues for render-space distillation and narrow the claim accordingly.
Why not just copy the planes, as Head3D does? Plane copying works when teacher and student planes are interchangeable. That holds in Head3D, where both generators share the same architecture. The gauge diagnostic tests whether the assumption fails in our setting, and the compression ladder tests whether the failure costs accuracy. If plane copying matches field-space distillation at every compression ratio, our central claim is wrong.
Why not just shrink the model and clean up its output with Freeplane's filtering? Freeplane shows that bilateral filtering of tri-plane features improves InstantMesh geometry without retraining. Chamfer Distance drops from 16.34 to 14.73. However, the same work notes that the filter erases thin structures and cannot restore information the model never learned. We will apply Freeplane to every student, including ours. If filtering a scratch student matches a distilled one, distillation is not buying knowledge, only smoothing.
Proposed Experiments
All students are evaluated on Google Scanned Objects (GSO) using the standard single-image protocol. We report PSNR, SSIM and LPIPS on novel views; Chamfer Distance, Volume IoU and Normal Consistency on extracted meshes; and latency and parameter count. In addition, each image-quality and geometry metric is computed separately on views and surface regions that were not visible in the input.
The decisive check is the compression ladder. It plots, for each distillation method, the fraction of the scratch-to-teacher gap that the method closes, against plane resolution and channel count. The gauge account predicts that the plane-space curve falls off faster than the field-space curve. A flat difference would falsify it.
A second check concerns the unseen regions. If the relational term matters, it should improve unseen-side LPIPS and Chamfer Distance more than visible-side metrics. It should also preserve thin structures that Freeplane's filter removes, which we will measure on a hand-labeled subset of GSO objects containing thin parts.
Finally, the gauge diagnostic reports the correlation between plane-space loss and Chamfer Distance across training checkpoints, which grounds the mechanism independently of the headline numbers.
References
- LRM: Large Reconstruction Model for Single Image to 3D
- Head3D: Complete 3D Head Generation via Tri-plane Feature Distillation
- Convolutional Occupancy Networks
- Relational Knowledge Distillation
- Freeplane: Unlocking Free Lunch in Triplane-Based Sparse-View Reconstruction Models
- Triplane Meets Gaussian Splatting: Fast and Generalizable Single-View 3D Reconstruction with Transformers
- Complex-YOLO: An Euler-Region-Proposal for Real-Time 3D Object Detection on Point Clouds
- Fused-Planes: Why Train a Thousand Tri-Planes When You Can Share?
- ControLRM: Fast and Controllable 3D Generation via Large Reconstruction Model
- PartDistill: 3D Shape Part Segmentation by Vision-Language Model Distillation
- Open-Vocabulary 3D Semantic Segmentation with Foundation Models
- PartSAM: A Scalable Promptable Part Segmentation Model Trained on Native 3D Data
- CF3: Compact and Fast 3D Feature Fields
- Google Scanned Objects: A High-Quality Dataset of 3D Scanned Household Items