Topographic Sheets as a Label-Free Dial for Compression in Semi-Supervised Learning

When a network has only a handful of labels per class, the hardest practical problem is often not training it but tuning it. Every regularizer has a strength that must be chosen. With 40 labeled images there is no trustworthy validation set to choose it on, and the variance of tiny validation sets makes model selection statistically unreliable (Realistic Evaluation of Deep Semi-Supervised Learning Algorithms). Information bottleneck regularization makes this especially painful. That approach trains a representation to keep what predicts the label and discard everything else. Its strength parameter is notoriously sensitive: set it slightly too high and useful features collapse, slightly too low and the model memorizes noise. Estimating the information quantities themselves also requires density estimates that do not scale beyond a few dimensions.

We propose to place the hidden units of a semi-supervised network on a two-dimensional "cortical sheet" and to penalize differences between neighboring units. This is a distance-dependent regularizer of the kind used to make networks develop brain-like maps. The point is that the sheet turns compression into something that can be measured and tuned without labels. The spatial smoothing scale acts as the bottleneck. And because each unlabeled image can be summarized by where on the 2D sheet it activates, the information trade-off can be computed exactly in two dimensions with classical clustering-based bottleneck methods, instead of being approximated in hundreds. The claim to test is simple: choosing the smoothing scale from the geometry of unlabeled activations on the sheet matches choosing it with a large labeled validation set, and an ordinary aspatial network offers no comparable label-free signal.

Core Claim

Topographic regularization has so far been studied as a route to biological realism under full supervision. TopoNets: High performing vision and language models with brain-like topography arranges a layer's weights on a 2D sheet, blurs the sheet by downsampling and upsampling, and rewards similarity between the raw and blurred sheet. The authors observe that this pushes representations onto a lower-dimensional, highly compressible manifold, but they treat the compression as an unquantified byproduct. End-to-end topographic networks as models of cortical map formation and human visual behaviour: moving beyond convolutions obtains category-selective patches on a sheet and explicitly flags full supervision on Ecoset as a limitation.

Our first claim is that the blur scale is a bottleneck parameter in the information sense: coarser blur removes high-spatial-frequency variation across units. Our second claim is that once the representation is topographic, its useful structure becomes low-dimensional enough to read off directly. If images of one category activate one contiguous patch, then the 2D peak-activation locations of unlabeled images form roughly $K$ blobs, where $K$ is the known number of classes.

The information bottleneck and geometric clustering supplies the method for reading these blobs. It applies the deterministic information bottleneck, a version that assigns each point to exactly one cluster, to points in Euclidean space through a Gaussian smoothing kernel. It shows that this reduces to hard Gaussian-mixture fitting, and it uses a "kink" in the curve of retained information versus cluster count to pick the number of clusters. That paper's tabular method is hopeless on raw network features, which is a limitation the file notes. It is directly usable on 2D sheet coordinates. We reuse it as a label-free selector rather than as a clustering tool.

Concretely, the network is trained on

$$\mathcal{L} = \mathcal{L}_{\text{FixMatch}} + \tau \,\mathcal{L}_{\text{topo}}(s),$$

where $\mathcal{L}_{\text{FixMatch}}$ is the standard semi-supervised loss from FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence. That loss trains the model on its own confident predictions from weakly perturbed unlabeled images and requires it to reproduce those predictions on heavily perturbed versions. $\mathcal{L}_{\text{topo}}(s)$ is the TopoLoss penalty with blur factor $s$, and $\tau$ is its weight.

For each candidate setting we map every unlabeled image $u$ to the sheet location $c(u)$ of its activation peak in the topographic layer. We then run deterministic-bottleneck geometric clustering on these 2D points and score the setting by

$$\kappa(s,\tau) = \Delta^2 I_k \big|_{k=K},$$

the sharpness of the kink at $k=K$ in the information curve $I_k$. Here $I_k$ is the information between cluster assignment and sheet location when $k$ clusters are used, and $\Delta^2$ is its discrete second difference in $k$. The selected setting maximizes $\kappa$. No labels enter this selection.

Research Program

The build is deliberately small. We use the Wide ResNet-28-2 backbone standard in semi-supervised work and arrange the channels of its last two stages on 2D sheets. We apply TopoLoss there over a grid of blur factors $s$ and weights $\tau$, from no topography to strong smoothing.

We also run a second variant using the explicit wiring-cost form from Growing Brains: Co-emergence of Anatomical and Functional Modularity in Recurrent Neural Networks. In that form, weight magnitudes between consecutive layers are penalized in proportion to the sheet distance between the units they connect. We omit its neuron-swapping step, which the file notes scales poorly. This second variant tests whether the effect depends on TopoLoss specifically or on distance-dependent regularization in general.

The analysis has three parts. First, we check that the blur scale behaves like a bottleneck. As $s$ grows, the effective dimensionality of sheet activity on unlabeled data should fall monotonically, and so should a variational upper bound on input information from a lightweight probe.

Second, we measure whether the kink score $\kappa$ tracks test accuracy across the grid. This is the central empirical question, and it can fail. Topography might help accuracy without producing $K$ clean blobs, or blobs might form around nuisance factors such as color.

Third, we test a mechanistic hypothesis tied to the known failure modes of pseudo-labeling. FixMatch suffers from confirmation bias when early pseudo-labels are wrong, and it degrades under class mismatch in the unlabeled pool (Realistic Evaluation of Deep Semi-Supervised Learning Algorithms). If the sheet organizes categories into patches, then out-of-distribution unlabeled images should peak between patches. We will measure whether pseudo-label precision is higher for in-patch than off-patch images, and whether discarding off-patch images reduces the damage from class mismatch.

This third part interacts with a real tension. Towards Understanding Why FixMatch Generalizes Better Than Supervised Learning argues that FixMatch helps precisely by forcing the network to learn more features than lazy supervised training does, for example headlights and mirrors as well as wheels. Compression pulls the other way. We therefore expect accuracy to be non-monotone in $s$, and we expect too much smoothing to undo FixMatch's multi-feature discovery. Selecting the right point on that curve without labels is the whole value of the proposal.

Baselines

The naive reference point is plain FixMatch on an aspatial Wide ResNet-28-2, using the paper's default hyperparameters with no extra regularizer. Its reported accuracy is 94.93% on CIFAR-10 with 250 labels, 88.61% with 40 labels, and a median of 64.28% with one label per class (FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence). We will rerun it under our own seeds and labeled splits rather than relying on the published numbers. Beating it means a higher mean accuracy across at least five labeled splits at 40 and 250 labels, with no larger spread across splits.

The strongest compression competitor is FixMatch plus a variational information bottleneck on the penultimate representation. That is the deep VIB loss described in the file, with a Gaussian encoder and a KL penalty against a standard normal prior, applied on the aspatial backbone. It is the right competitor because it targets the same goal, discarding input information the label does not need. It is also the method that worked dramatically on shortcut-laden ultrasound scans: Texture Synthesis Based Thyroid Nodule Detection From Medical Ultrasound Images: Interpreting and Suppressing the Adversarial Effect of In-place Manual Annotation reports clean-scan detection rising from 27.49% to 89.61% mAP with inpainting plus VIB.

For fairness, VIB's strength receives the same grid size and the same training budget as our $(s,\tau)$ grid. Its strength is selected in three ways: by a realistic validation set of the same size as the labeled set, by a large oracle labeled set, and by the label-free selectors described next. Beating it means that our label-free selection achieves lower regret than VIB under any selection rule that also avoids extra labels. Regret here is the gap to the oracle-selected setting.

The ablation that isolates the key ingredient keeps the trained topographic network fixed but randomly permutes unit positions before computing $\kappa$. This preserves every activation and destroys only the geometry. A second ablation computes the same kink criterion with k-means in the full aspatial feature space. If either ablation selects as well as the true sheet, the spatial arrangement is doing no work.

Why not just…?

Why not just tune the regularizer on the small validation set? Because that is exactly what Realistic Evaluation of Deep Semi-Supervised Learning Algorithms shows to be unreliable at realistic sizes. We do not assume this; the realistic-validation selection rule is run for every method, and its regret is reported next to ours. If it turns out adequate on our benchmarks, the selector contributes nothing and we will say so.

Why not just select hyperparameters with a label-free signal that needs no topography, such as the unlabeled consistency loss or the fraction of pseudo-labels above the confidence threshold? This is the most serious threat. Such signals are cheap and work on any backbone. They are also known to be fooled by confident wrong pseudo-labels, which is the confirmation-bias problem FixMatch itself acknowledges. We will apply both signals to select VIB strength and to select $(s,\tau)$, and compare their regret against $\kappa$. If consistency-based selection on an aspatial network matches $\kappa$, the honest conclusion is that topography is unnecessary for label-free tuning, and the proposal reduces to a negative result about topographic sheets.

Why not just fine-tune an ImageNet-pretrained model? Realistic Evaluation of Deep Semi-Supervised Learning Algorithms reports 12.09% CIFAR-10 error from fine-tuning a pretrained model, beating every semi-supervised method it trained from scratch. We do not claim to beat pretraining. Our question is about tuning compression when no appropriate pretrained model exists, as with specialized ultrasound. We will still report a pretrained fine-tuning reference on CIFAR-10 so readers can see the gap. We will also test whether the sheet and selector can be added on top of a pretrained backbone during fine-tuning.

Why not just use the topographic loss at a fixed published setting? TopoNets reports only about a 3% ImageNet drop at its chosen setting under full supervision (TopoNets: High performing vision and language models with brain-like topography). That setting was tuned with abundant labels, and earlier topographic methods lost 11% (credit-based self-organizing maps) to as much as 41% (Credit-based self organizing maps: training deep topographic networks with minimal performance degradation). That spread of costs shows how much the strength matters. We include the transferred default setting as a selection rule and measure its regret.

Proposed Experiments

The main benchmarks are CIFAR-10 at 40 and 250 labels and CIFAR-100 at its standard low-label splits. We add STL-10, whose unlabeled pool contains classes absent from the labeled set, and the CIFAR-10 class-mismatch protocol of Realistic Evaluation of Deep Semi-Supervised Learning Algorithms. There, consistency methods fell below a purely supervised baseline as mismatch grew.

As a shortcut test, we use the thyroid ultrasound dataset with clinician crosshairs, keeping 5% of images labeled and the rest unlabeled. We know of no reported low-label result for it on Lacuna, so the fully supervised VIB numbers above serve only as an upper anchor.

For every method and selection rule, we report test accuracy (mAP on ultrasound), regret relative to oracle selection, and spread across five labeled splits and three seeds. We also report pseudo-label precision over training, split by in-patch versus off-patch images.

Three results would decide the proposal. The first is a rank correlation between $\kappa$ and test accuracy across the $(s,\tau)$ grid that clearly exceeds the correlation achieved by the consistency-loss and confidence selectors. The second is that this correlation disappears when unit positions are permuted. The third is that label-free selection on the sheet beats realistic-validation selection of VIB in regret at 40 labels.

If all three hold, topography earns a concrete role under label scarcity: it makes the bottleneck observable. If the first holds but the second fails, the effect comes from compression, not geometry. If the first fails, the sheet does not organize unlabeled data into category patches at these label budgets. That would itself be the first measurement of how topographic maps behave when supervision is sparse, a question the All-TNN and TopoNets work leaves open.