Frozen-Pairing Topological Distillation for Point Clouds

Teaching a small 3D segmentation network to copy the shape of a large network's internal representation works well, and it is expensive. Topology-guided distillation does this by comparing persistence diagrams. A persistence diagram is a summary of which clusters and loops appear and disappear as you grow balls around points in feature space. The problem is that these diagrams are recomputed for both teacher and student at every training step. Our observation is that the costly part of a persistence diagram is not the numbers in it but the pairing: which specific pairs of points create and destroy each cluster or loop. Once the pairing is known, every value in the diagram is just a distance between two points. Gradients through persistence already flow only through these critical pairs.

We therefore propose to compute the frozen teacher's critical pairs once, offline. Online, the student is trained to reproduce the teacher's distances on exactly those pairs. The structural distillation step becomes a gather over cached indices followed by a few thousand distance computations, with no filtration at training time. The proposal turns the precompute-and-index idea from 3D training pipelines into a distillation loss. It also says which part of topological knowledge can be cached exactly (the teacher side) and which part must be approximated (the student's own pairing).

Core Claim

For a frozen teacher, the 0- and 1-dimensional persistence pairing of its token features on a canonical, unaugmented scan is a fixed property of the dataset. It can be stored compactly. For the 0-dimensional case it is just the minimum spanning tree edges, which is $m-1$ index pairs for $m$ tokens. A student matched to the teacher's birth and death distances on these cached edges should recover most of the benefit of full online topological distillation, at a small fraction of its training cost.

The loss is

$$ \mathcal{L}_{fp} = \sum_{(i,j)\in \mathcal{P}_T} w_{ij}\,\big(\lVert s_i - s_j\rVert - \lVert t_i - t_j\rVert\big)^2 . $$

Here $\mathcal{P}_T$ is the cached set of teacher critical pairs, made of index pairs of tokens. The vectors $s_i$ and $t_i$ are the student and teacher features of token $i$, with the student features passed through a learned projection. The weight $w_{ij}$ is the persistence (death minus birth) of the feature that pair belongs to, so that long-lived structure dominates.

Seen another way, this is pairwise-distance relational distillation restricted to the pairs that topology says matter. The central empirical question follows directly: do those pairs beat random or nearest-neighbor pairs at an equal budget?

Relation to Prior Work

The method's motivation and its strongest competitor come from structured 3D distillation. PointDistill compresses Point Transformer V3 using Vietoris-Rips persistence diagrams matched by Chamfer distance, together with gradient-weighted channel alignment. It computes the diagrams online during training and reports sensitivity to the filtration scale. We keep its goal and teacher and replace its online filtration with cached pairings.

PointDistiller builds local nearest-neighbor graphs and dynamic graph convolutions during training to distill local geometry for KITTI detectors. It discards these layers at test time but still rebuilds the graphs at every training step. Its local graphs are the natural non-topological pair set to ablate against.

Relational Knowledge Distillation supplies the loss form, matching pairwise distances. It chooses pairs without regard to structure; our contribution is the pair selection and its offline computation.

The decoupling technique comes from 3D training efficiency work. PointHR caches nearest-neighbor indices on static coordinates and trains about 40% faster than computing them on the fly. Frugal 3D training replaces repeated point sampling with precomputed bitwise masks and reaches 2.25× faster training. Both cache geometry for ordinary supervised training. Neither caches a teacher-derived signal.

Frugal 3D training also imposes a constraint. Its caching caused a 1.16× slowdown on ModelNet40, where loading from disk outweighed computing on the fly. Caches must therefore be small index tables rather than dense tensors, which is why we store pairs and scalar distances rather than teacher features.

The closest operational relatives split distillation into offline and online parts, but along different axes:

Neither caches structural targets. SHAKE does constrain us: freezing the teacher's targets rules out its feedback from student to teacher. We accept this, because PointDistill is itself a frozen-teacher method, and we name SHAKE-style adaptation as an extension rather than claim to match it.

Research Program

Offline stage. We run the frozen teacher once per training scan on the canonical, unaugmented input, and take token features at the stage PointDistill distills. On those tokens we compute Vietoris-Rips persistence in dimensions 0 and 1 with a standard library. We store the critical index pairs, their teacher distances and their persistence weights. Storage grows linearly in the token count for dimension 0 and is typically much smaller for dimension 1. We will report bytes per scan and loading time against compute time, following the Frugal 3D methodology.

Online stage. The student sees augmented inputs: rotation, flipping, scaling and point dropout. Rigid transforms preserve point identity. Under dropout we keep only cached pairs whose endpoints survive. Student tokens are matched to teacher tokens through the shared voxel and token indexing.

This creates a deliberate asymmetry: the teacher targets come from the clean scan while the student sees an augmented one. That may act as a useful consistency signal or may hurt. We test it directly against computing teacher pairings on the augmented scan.

Staleness. Frozen pairing ignores structure the student invents that the teacher lacks, such as spurious loops. To measure this, we periodically compute the student's true persistence diagram on a held-out subset. We track its bottleneck distance to the teacher diagram over training and compare it with a variant that recomputes student pairings every $k$ epochs.

Baselines

Naive reference. The do-nothing reference is the PointDistill student architecture (2.78M parameters) trained with ground truth only. It also indicates how much of any distillation gain is structural.

Strongest existing method. The strongest competitor is PointDistill itself on nuScenes LiDAR segmentation. It reports 78.17% mIoU for the 2.78M-parameter student distilled from a 46.16M-parameter Point Transformer V3 teacher (PointDistill). We found no reported result on Lacuna for PointDistill's training wall-clock time or per-step cost, so we will measure it ourselves.

Fair comparison protocol. We reimplement PointDistill's topological loss and its gradient-guided alignment. Both methods share:

  • the same teacher checkpoint and student;
  • the same augmentation and schedule;
  • the same token stage and the same hyperparameter search budget over loss weights.

Our method keeps the gradient-guided alignment term, so the only difference is how the topological term is computed. We beat PointDistill if we land within 0.5 mIoU of it on nuScenes and SemanticKITTI while cutting wall-clock time per training step and peak training memory substantially. We report the offline compute separately, amortized over epochs.

Logit baseline. Decoupled Knowledge Distillation separates target-class and non-target-class terms of the logit loss. It adds no parameters and reaches 71.70% ImageNet top-1 with a ResNet-18 student. We apply it per point. It is the cheap logit-only competitor, and teacher logits can be cached as well.

Isolating ablation. The ablation that isolates the key ingredient keeps the loss above fixed and changes only the pair set, holding the number of pairs equal:

  • teacher critical pairs;
  • uniformly random pairs, which is plain relational distillation;
  • local nearest-neighbor pairs in the spirit of PointDistiller;
  • teacher pairs with persistence weights replaced by uniform weights.

Why not just…?

Why not just cache teacher logits and use decoupled logit distillation? This is cheapest, and if it matches PointDistill on segmentation the structural direction is moot for this task. Two things argue against it, neither decisive. DKD's own evidence is on classification. PointDistill reports beating other LiDAR distillation baselines when it adds structure. The proposal runs per-point DKD with cached logits as a full baseline. If DKD matches PointDistill, we report that as the main finding.

Why not just use relational distillation on random or neighboring pairs? This is the real threat. Our loss is relational distillation with chosen pairs, so if random pairs at the same budget do as well, the topological selection adds nothing. The pair-set ablation settles this directly. We expect the difference to show most on thin and sparse classes such as poles and bicycles, where cluster connectivity is fragile, so we report per-class IoU as well as mean.

Why not just run online persistence on a heavily subsampled token set? Subsampling until filtration is cheap is a legitimate alternative to caching. It keeps the student's own pairing and loses resolution. We include it, using subsampled online PointDistill at matched step cost. This shows whether cached full-resolution pairs beat fresh low-resolution ones.

Proposed Experiments

The primary benchmarks are nuScenes and SemanticKITTI LiDAR segmentation with the PointDistill teacher and student, measured by mIoU. Efficiency metrics are training time per step, peak training memory, offline preprocessing hours, and cache size.

The first decisive check is the parity-at-lower-cost comparison with PointDistill described in Baselines.

The second is the pair-set ablation. If critical pairs do not beat random pairs by a margin larger than seed variance, measured over three seeds, the topological claim fails.

The third is the staleness study. If the student's true diagram drifts from the teacher's without refreshing, and refreshing every $k$ epochs closes the accuracy gap, the method becomes "lazy" topology rather than frozen topology. That is still a useful, cheaper point on the curve.

The fourth is the I/O stress test on a small dataset such as ModelNet40. It checks whether index-only caches avoid the slowdown that Frugal 3D training observed with denser caches.

A secondary transfer test applies the cached nearest-neighbor and critical-pair targets to PointDistiller's KITTI PointPillars setting. PointDistiller reports +3.4 3D mAP for a 4× compressed student. The test asks whether caching preserves that gain.