Neural networks that classify signals living on a sphere, such as 360° panoramas, global climate fields or wind maps, come in two kinds. Isotropic networks treat every direction around a point the same way. Anisotropic networks use direction-aware filters that can tell "up" from "sideways" on the curved surface. Anisotropic filters are needed whenever orientation carries the label. On a rotated benchmark of vector fields drawn on the sphere, an isotropic model reaches 24.8% accuracy while a spin-weighted anisotropic model reaches 98.2% (Spin-Weighted Spherical CNNs). The question here is what happens to these networks under adversarial training, which retrains a model on worst-case perturbed inputs so that small changes cannot flip its prediction. On flat images, adversarial training is known to cost clean accuracy (Theoretically Principled Trade-off between Robustness and Accuracy). Nobody has asked which features of a spherical network pay that cost.
Our hypothesis is that, on the sphere, the clean accuracy lost to adversarial training is mostly orientation information. Spin-weighted networks make this testable, because they already sort every hidden feature by its "spin": spin-0 channels carry orientation-free content, and nonzero-spin channels carry directional content. We propose to measure, spin by spin, where robust training removes feature energy and which test decisions it breaks. We then test whether a robustness penalty with separate strengths for directional and non-directional channels gives a better trade-off than the usual single global knob.
Core Claim
The claim has two parts. The first is a measurement claim. When a spin-weighted spherical network is trained with TRADES, it suppresses nonzero-spin features more strongly than spin-0 features. TRADES, from Theoretically Principled Trade-off between Robustness and Accuracy, adds to the usual classification loss a penalty for any change in the model's output between a clean input and its worst-case perturbation, weighted by one coefficient $\lambda$. We expect the clean errors it introduces to concentrate on decisions that depend on orientation, such as telling a 6 from a 9 or classifying a vector field.
The second is a method claim. Directional channels may be both the most useful and the most fragile. If so, giving them their own robustness weight should move the clean-versus-robust Pareto front outward compared with tuning the single global $\lambda$. Under the same robustness, the model should keep more clean accuracy.
Either part can fail independently. If adversarial training removes energy uniformly across spins, or if splitting the weight by spin does no better than an equally flexible split of channels into random groups, the hypothesis is wrong. That outcome would itself be informative. It would mean the robustness-accuracy trade-off on the sphere is indifferent to filter anisotropy, which would strengthen DeepSphere: a graph-based spherical CNN and its argument that directional filters are often an unnecessary price.
Research Program
A sphere-consistent threat model. The usual attack bounds the perturbation of each pixel of the equirectangular image, the flat latitude-longitude unwrapping of the sphere. Pixels near the poles of that image cover tiny patches of the sphere, so a per-pixel bound over-perturbs the poles and under-perturbs the equator. Geometry Aware Convolutional Filters for Omnidirectional Images Representation documents exactly this distortion as a failure mode of flat CNNs. We therefore measure perturbation size with an area-weighted norm, $\|\delta\|_{S^2}^2=\sum_i w_i\,\delta_i^2$. Here $\delta_i$ is the perturbation at sample point $i$ and $w_i$ is the quadrature weight of that point, which on an equirectangular grid is proportional to the cosine of its latitude. We report every result under both this norm and the naive pixel norm, so that the comparison with flat-image robustness work stays legible.
Spin-resolved diagnostics. Spin-weighted networks store each intermediate feature map $h$ as a sum of components $P_s h$, one per spin weight $s$, where $P_s$ is the exact projection onto spin $s$. This gives us, for free, a decomposition that flat CNNs lack.
We track two quantities across training and across a sweep of $\lambda$. The first is the energy fraction of each spin, $\|P_s h\|^2/\|h\|^2$. The second is an orientation-sensitivity split of the test errors: each test sample is labelled orientation-dependent if its clean label differs from the label of its in-plane 180° rotated version, as with 6 and 9, and orientation-free otherwise. Hypothesis H1 predicts that, as $\lambda$ grows, two things happen together. The nonzero-spin energy fraction falls faster than the spin-0 fraction, and the new clean errors fall disproportionately in the orientation-dependent subset.
Spin-selective TRADES. We keep TRADES's inner attack unchanged: the perturbed input $x^*$ maximizes the output disagreement within the threat ball. We change only the outer penalty, adding a normalized feature-consistency term for each spin group at an intermediate layer:
$$\min_f\ \mathbb{E}\Big[\mathcal{L}(f(x),y)+\lambda\,\mathrm{KL}\big(f(x)\,\|\,f(x^*)\big)+\sum_{g\in\{0,\mathrm{dir}\}}\gamma_g\,\frac{\|P_g h(x)-P_g h(x^*)\|^2}{\|P_g h(x)\|^2}\Big]$$
In this objective, $\mathcal{L}$ is cross-entropy, $g=0$ selects the spin-0 channels, $g=\mathrm{dir}$ selects all nonzero-spin channels, and $\gamma_g$ is the robustness weight for that group. Restricting the split to two groups keeps the tuning burden comparable to plain TRADES. The variant with $\gamma_{\mathrm{dir}}$ set to zero is the extreme in which directional features are left free and only orientation-free features are made robust.
The perturbation is generated exactly as in TRADES. So the only ingredient that is new is where in the representation the robustness pressure is applied, and that is what the ablations isolate.
Relation to prior work. TART: Boosting Clean Accuracy Through Tangent Direction Guided Adversarial Training motivates this design. It recovers clean accuracy on CIFAR-10 by applying adversarial training only to perturbations that lie along an estimated data manifold, which it builds from an autoencoder and local PCA.
We do not reuse TART's manifold estimate, and we avoid a tempting confusion. The tangent plane of the sphere at a point is not the tangent space of the data distribution. The sphere's geometry tells us nothing directly about which perturbations are "on-manifold" for the data. What the geometry does supply is an exact, closed-form sorting of features by directional content. Our proposal acts on that sorting, inside the network, rather than on the input perturbation.
Certified Adversarial Robustness via Anisotropic Randomized Smoothing uses "anisotropic" in a different sense: it learns a separate noise variance for each pixel for certification on flat grids. We treat certified robustness as out of scope, and discuss the reason below.
SAGA: Spectral Adversarial Geometric Attack on 3D Meshes constrains our evaluation. It shows that smooth, low-frequency perturbations defeat low-pass defenses on mesh autoencoders. We therefore add a low-degree spherical-harmonic attack, which perturbs only the coarse harmonic coefficients, to the test suite. This prevents a model from looking robust merely because its training happened to smooth away high frequencies.
Baselines
Naive reference. The do-nothing reference is the same spin-weighted architecture as Spin-Weighted Spherical CNNs, trained without any robustness objective. It anchors clean accuracy; that paper reports 99.37% on rotated Spherical MNIST and 98.2% on the rotated vector-field benchmark (SVFMNIST). We retrain it under our protocol so that its adversarial accuracy is measured under the same attacks as every other model.
Strongest existing competitor. The strongest competitor is TRADES on that same architecture, sweeping only the global $\lambda$. It is the method whose single knob we claim is too coarse. It is also the strongest documented trade-off controller in the file: on CIFAR-10 it reports 56.61% PGD robust accuracy and spans up to 88.64% natural accuracy as $\lambda$ varies (Theoretically Principled Trade-off between Robustness and Accuracy).
No reported result was found on Lacuna for any adversarially trained spherical classifier on Spherical MNIST, SVFMNIST or ModelNet40. Our TRADES runs will therefore themselves establish the reference trade-off curve on the sphere.
For fairness, global TRADES receives the same number of training runs as our two-weight method. That means a denser one-dimensional grid of $\lambda$ values, with identical data, attack steps, epochs and model size. Our method is counted as beating TRADES only if its Pareto front lies above TRADES's at matched robust accuracy, by more than the seed-to-seed spread over three seeds, on at least the two orientation-heavy benchmarks.
Isotropic adversarially trained network. This baseline is a DeepSphere graph network (DeepSphere: a graph-based spherical CNN) trained with TRADES. It tests the claim that isotropic smoothing is already a regularizer and avoids directional fragility.
Flat baseline. This is a standard 2D CNN trained with TRADES on the equirectangular projection. It measures the cost of ignoring spherical geometry altogether.
Key ablation. The key ablation replaces the spin split with a random split of channels into two groups of the same sizes, using the same two-weight tuning budget. A second variant uses a learned split of channels into groups, in the spirit of Pixel Reweighted Adversarial Training, which reweights the perturbation by input importance. If either variant matches the spin split, then the extra tuning freedom, not the anisotropy, explains any gain.
Why not just…?
Why not just tune the global TRADES weight harder? This is the most serious threat, and the main experiment settles it directly. Global TRADES gets a larger one-dimensional grid than our method gets in two dimensions. If its front matches ours, the spin-selective method is unnecessary, and we would report the diagnostic study alone. The diagnostic study is still new, because no one has located spherical trade-off losses by spin.
Why not just use an isotropic network, which is smoother and perhaps naturally more robust? On orientation-dependent tasks, an isotropic network gives up accuracy before any attack happens. The 24.8% versus 98.2% gap on rotated SVFMNIST (Spin-Weighted Spherical CNNs) and the 19.8% versus 0.60% error gap on Embedded MNIST (Gauge Equivariant Mesh CNNs: Anisotropic convolutions on geometric graphs) show this. Smoothing is also no guarantee against smooth attacks, as SAGA shows on meshes. On orientation-free tasks, however, the isotropic baseline may honestly win. We will report that case rather than hide it, because it maps out where anisotropy is worth its robustness cost.
Why not just estimate a data manifold, as TART does, and restrict the attack to it? TART's autoencoder-plus-PCA pipeline is tied to stored tangent bases and does not work with on-the-fly augmentation (TART), and random rotation is the central augmentation for spherical data. TART also addresses a different axis, namely which input perturbations to train on, whereas we address which internal features to make robust. The two can be combined. We include TART on the spin-weighted model as a secondary baseline only on the unrotated Spherical MNIST split, where its pre-computation is feasible.
Why not just use anisotropic randomized smoothing for provable guarantees? Randomized smoothing certifies robustness by averaging the classifier's predictions over random input noise. Anisotropic Randomized Smoothing assumes independent noise at each flat pixel, and its certified radius is set by the smallest-variance dimension. On an equirectangular grid, that dimension is entangled with latitude distortion. Extending the method to the sphere is a separate problem. Our study first establishes whether the empirical trade-off depends on spin at all, which would guide how such noise should later be shaped.
Proposed Experiments
We use four benchmarks, each with a distinct role. Rotated Spherical MNIST is a scalar signal with a built-in orientation-dependent subset (6 and 9). SVFMNIST is a vector field, where directional content is essential. Modified Spherical MNIST, specifically its most distorted split MS3, tests whether the effects survive non-spherical distortion; on MS3, Geometry Aware Convolutional Filters reports 94.9% for its filters against 80.9% for SphericalCNN. ModelNet40 is a realistic check, with 90.1% clean accuracy reported for the spin-weighted network.
The attacks are PGD under the area-weighted norm, PGD under the pixel norm, and the low-degree harmonic attack. The metrics are clean accuracy, robust accuracy, the clean-versus-robust Pareto front over the full sweep, the energy fraction of each spin, and error rates on the orientation-dependent and orientation-free subsets.
Three checks decide the outcome. The first checks hypothesis H1. Under global TRADES, the orientation-dependent error rate should rise faster with $\lambda$ than the orientation-free rate, and nonzero-spin energy should fall faster than spin-0 energy. If neither holds, the premise fails.
The second check asks whether the gain comes from the spin split itself. The spin-selective Pareto front should exceed both the densely tuned global TRADES front and the random-split ablation. A win over TRADES alone would not be enough.
The third check is a reversal test. On orientation-free content, meaning ModelNet40 and the orientation-free subset of Spherical MNIST, the advantage of spin-selective weighting should shrink toward zero. Such a predicted reversal would be hard to explain by tuning freedom alone.
As a side result, we report how much the pixel-norm and area-weighted-norm rankings of the models disagree. This tells future spherical robustness work whether the flat threat model can be trusted at all.
References
- Spin-Weighted Spherical CNNs
- Theoretically Principled Trade-off between Robustness and Accuracy
- DeepSphere: a graph-based spherical CNN
- Geometry Aware Convolutional Filters for Omnidirectional Images Representation
- TART: Boosting Clean Accuracy Through Tangent Direction Guided Adversarial Training
- Certified Adversarial Robustness via Anisotropic Randomized Smoothing
- SAGA: Spectral Adversarial Geometric Attack on 3D Meshes
- Improving Accuracy-robustness Trade-off via Pixel Reweighted Adversarial Training
- Gauge Equivariant Mesh CNNs: Anisotropic convolutions on geometric graphs