Does It Matter Where in the Network Untangling Happens?

Deep networks gradually turn tangled inputs into representations that a straight line can separate. In raw pixels, the many images of one object category, seen under different poses, lighting and backgrounds, form curved, overlapping surfaces. Layer by layer, a trained network flattens and shrinks these surfaces until the categories pull apart. This process is called manifold untangling. Statistical mechanics gives it a precise score: manifold capacity, the number of category "clouds" per neuron that a linear classifier can still split into arbitrary groups. Capacity can be broken down into three geometric causes: how spread out each cloud is (its radius), how many directions it extends in (its dimension), and how its center and axes line up with those of other clouds. Researchers have plotted these quantities across the depth of image, speech and language networks and across the primate visual cortex.

All of this work treats the depth profile as a description. Nobody has tested whether its shape does anything. This proposal asks one causal question. Hold fixed the capacity a network reaches at its last layer. Does it then matter for robustness to distribution shift, and for keeping continuous information such as pose, whether untangling happens early or late in depth, and whether it comes from shrinking radius, shrinking dimension, or decorrelating centers? To answer it, we build a cheap, validated capacity estimate that can be computed at every layer and trained through. We use it to steer networks onto chosen depth profiles while matching their final-layer capacity, and then measure what changes.

Core Claim

We hypothesize that final-layer capacity does not summarize a network's useful geometry. Two networks with equal final capacity and equal in-distribution accuracy can differ systematically in how well they handle corrupted inputs, and that difference is predicted by the depth profile and by the geometric route that produced the capacity. Three findings motivate this.

First, Geometry Linked to Untangling Efficiency uses correlated manifold capacity theory to show that the macaque ventral stream untangles in different ways at different stages. The step from V1 to V2 is mostly radius shrinkage. The step from V4 to IT combines shrinking radius, shrinking dimension and reduced alignment. That suggests the route is organized, not incidental.

Second, Feature Learning beyond the Lazy-Rich Dichotomy finds an "ultra-rich" regime in which training accuracy stays high but performance on CIFAR-10C collapses. The collapse coincides with radius expansion and stronger center-axis alignment. That study tracked the final layer only.

Third, Neural Feature Geometry Evolves as Discrete Ricci Flow reports a critical depth beyond which layers stop reshaping neighborhood geometry. This raises the question of whether late, compressed untangling is fragile.

If the hypothesis is right, robustness is partly a property of where geometry is built, and it can be designed. If it is wrong, final capacity is a sufficient summary. That would justify the field's habit of regularizing only the output. Both outcomes are informative.

Research Program

The first component is a per-layer capacity estimate that is cheap enough to compute during training. Exact mean-field manifold analysis, as in Classification and Geometry of General Perceptual Manifolds and Separability and geometry of object manifolds in deep neural networks, solves coupled nonlinear equations per manifold. Maximum Manifold Capacity Representations notes that this is too heavy for an inner training loop. That paper avoids the cost by approximating each manifold as an ellipsoid made of augmented views. Under this approximation, maximizing capacity reduces to maximizing the nuclear norm (the sum of singular values) of the matrix of view centroids. MMCR in State Representation Learning shows the approximation still works with only two views.

We borrow that approximation but keep its parts separate rather than collapsing them into one number. At layer $l$, we form manifolds from the views of each sample, or from the samples of each class in the supervised setting. We then compute three terms on the minibatch:

  • a center term $c_l$, the nuclear norm of the normalized centroid matrix;
  • a radius term $r_l$, the within-manifold variance divided by the squared centroid norm;
  • a dimension term $d_l$, the participation ratio of the within-manifold covariance.

These combine into a surrogate capacity $s_l = g(c_l, r_l, d_l)$. We fit the function $g$ once, offline, by regressing exact mean-field capacity on the three terms across layers and checkpoints of reference networks.

Before any intervention, the estimate must pass validation. Its rank correlation with exact capacity across layers and training time must be high. It must also keep that correlation at the small widths of early layers, which matters because Soft-margin classification of object manifolds warns that the theory's large-width assumptions can fail there.

The second component is profile steering. We train with

$$\mathcal{L} = \mathcal{L}_{\text{task}} + \lambda \sum_{l} \left(s_l - s_l^{*}\right)^2,$$

where $\mathcal{L}_{\text{task}}$ is cross-entropy or an MMCR-style self-supervised loss, $s_l^{*}$ is the target capacity for layer $l$, and $\lambda$ weights the constraint. We define four profile families, all sharing the same final value $s_L^{*}$:

  • natural, the profile plain training produces;
  • early-loaded, where most capacity is reached by mid-depth;
  • late-loaded, which stays flat and then rises sharply;
  • linear, an even rise across depth.

To vary the route, we hold the profile fixed and change which term carries the target. Capacity is pushed up through the radius term, the dimension term, or the center term, and the other two are constrained to stay near their natural values.

The third component is post-hoc measurement with exact tools, so that conclusions do not rest on our own estimate. For every trained network we run exact mean-field analysis on held-out data and measure radius, dimension, center alignment and center-axis alignment, following Feature Learning beyond the Lazy-Rich Dichotomy. This lets us check two things: that the realized profile matches the intended one, and that final capacity really is matched across conditions.

Baselines

The naive reference point is plain cross-entropy training of the same architectures, ResNet-18 and VGG-16 on CIFAR-10 and CIFAR-100. We use the same augmentation, epochs and learning-rate schedule as every other condition, with five seeds. This defines the natural profile and the seed-noise floor that every difference must exceed.

The strongest competitor that requires no theory is deep supervision. Here auxiliary linear cross-entropy heads are attached at every block, with per-layer weights tuned so that layer-wise linear decodability follows each of our target profiles. It is the most serious threat because it can plausibly move the depth profile without any capacity theory at all. Its head weights receive the same tuning budget as our $\lambda$ and target schedule: an equal number of search trials, selected on in-distribution validation accuracy only.

The strongest published regularizer acting on hidden layers is Manifold Mixup, which interpolates the hidden states of pairs of examples at a randomly chosen layer. We run it with the official code, vikasverma1077/manifold_mixup. As a calibration check, our reproduction should approach the reported CIFAR-100 PreActResNet34 test error of 18.35%, against 23.55% for the plain baseline in that paper. Its reported CIFAR-10 FGSM accuracy of 77.50%, against 36.32%, gives a robustness reference point.

The ablation that isolates the key ingredient applies the capacity term at the final layer only. This is the MMCR-style choice made by MMCR, DIM-C+ and Token Manifold Capacity Learning. We tune it to reach the same final capacity as the profile-steered runs.

For the self-supervised arm on STL-10, the comparison points are MMCR and Barlow Twins, with and without the C+ regularizer. The file reports "90.6% accuracy" for Barlow Twins with C+, against 88% for standard Barlow Twins (MMCR in State Representation Learning). For CIFAR-10C under any of these methods, no reported result found on Lacuna for a scalar corruption accuracy.

What counts as winning depends on the comparison. A profile-steered condition "beats" final-layer-only capacity if it improves mean CIFAR-10C accuracy beyond seed noise while matching in-distribution accuracy and exact final capacity. It beats deep supervision only if steering through the geometric route explains variance in robustness that matched decodability profiles do not.

Why not just…?

Why not just train with cross-entropy, since it already untangles? It does. Separability and geometry of object manifolds shows supervised training produces the orchestrated drop in radius and dimension that random networks lack. Cross-entropy gives one profile, though, and the question here is whether other profiles at the same endpoint generalize differently. Plain training is the natural-profile condition, not a rival answer.

Why not just put auxiliary classifier heads at every layer? This is a real threat. If tuned deep supervision produces the same layer-wise capacity profile and the same CIFAR-10C behavior as our estimate, then the profile matters but the capacity machinery is unnecessary for steering it. We would report that honestly. The place where deep supervision should fall short is the route experiments. A linear decoding loss cannot ask for capacity gained through radius rather than center decorrelation. Geometry Linked to Untangling Efficiency shows that decoder accuracy can miss structure that capacity detects: on THINGS fMRI, a support vector machine found no body-versus-face difference where capacity analysis found one.

Why not just use Manifold Mixup? It flattens hidden representations cheaply and has strong reported gains. However, it picks a random layer per batch and has no target profile. It therefore cannot test whether where flattening happens matters. We include it as a baseline and measure its realized profile exactly. If its robustness gains are explained by the profile it happens to induce, that supports our claim rather than undercutting it.

Why not just maximize capacity at the output, as MMCR does? That is precisely the hypothesis under test, and the final-layer-only ablation measures it directly. The ultra-rich result in Feature Learning beyond the Lazy-Rich Dichotomy suggests that output-only pressure can hide harmful geometry. Whether that happens at matched final capacity is an open empirical question.

Proposed Experiments

The first decisive check validates the estimate. We report its rank correlation with exact capacity per layer and per checkpoint on CIFAR-100 and on an ImageNet-100 subset. If the correlation is weak in narrow early layers, we restrict steering to layers where it holds.

The main experiment trains the four profile families and the three routes on CIFAR-10 and CIFAR-100, with five seeds each. We report:

  • in-distribution accuracy;
  • mean CIFAR-10C corruption accuracy;
  • FGSM accuracy;
  • a continuous-information probe: the R² of a linear regression of applied rotation angle from penultimate features. This tests the pose preservation documented by Digging Deep into the layers of CNNs.

The analysis regresses CIFAR-10C accuracy on exact final capacity alone, then on final capacity plus profile shape and route. The hypothesis is supported if the added terms explain variance beyond seed noise. It is refuted if final capacity alone suffices.

Specific predictions:

  • Late-loaded profiles will reproduce the ultra-rich signature, meaning radius expansion and center-axis alignment, along with weaker corruption robustness.
  • Radius-led early untangling will best preserve rotation information.
  • Center-led routes will gain in-distribution accuracy but not robustness.

A smaller transfer check repeats the comparison of the natural, early-loaded and final-layer-only conditions in the MMCR self-supervised setting on STL-10, which tests whether the effect survives without labels.