Does Complex Structure Soften the Robustness–Accuracy Trade-Off?

Adversarial training teaches a network to keep its prediction when an attacker adds a small, worst-case perturbation to the input. It reliably costs accuracy on ordinary, unperturbed inputs. Much sensor data (radar returns, Fourier spectra, audio spectrograms) is complex-valued: each number has a magnitude and a phase. Such networks are usually built by splitting each complex number into its real and imaginary parts and processing them with ordinary real-valued layers. One such design, the Analytic Neural Network, processes the two parts in separate real subnetworks and adds a penalty forcing their hidden features to be related by the Hilbert transform. The Hilbert transform is a fixed signal-processing operator that pairs a signal with its 90-degree phase-shifted partner, so penalizing departures from it forces the two feature streams to act as the real and imaginary parts of one coherent complex signal.

We propose using that Hilbert relation as the regularizer in adversarial training. The constraint applies to any input regardless of its label. Pulling attacked features back toward it should therefore fight the classification objective less than today's method, which pulls attacked features toward clean ones. We also give a width-controlled test of whether any gain comes from genuine complex structure or merely from having twice as many real units.

Core Claim

The strongest current fix for the lost clean accuracy is Asymmetric Representation-regularized Adversarial Training (AR-AT), from Rethinking Invariance Regularization in Adversarial Training. It traces the loss to two causes. The first is gradient conflict: a term forcing clean and attacked features to match pulls against the classification loss. The second is a mixture problem: batch normalization estimates a single set of statistics from batches that contain two different distributions, clean and attacked. AR-AT addresses the first cause with a one-way regularizer, which blocks gradients through the clean features and adds a small predictor head. It addresses the second with Split-BatchNorm, meaning separate normalization statistics for clean and attacked samples.

Both of AR-AT's remedies are built around class-specific feature matching and batch normalization. Neither exists naturally in the decoupled two-path networks of Steinmetz Neural Networks for Complex-Valued Data. Those networks also have only been tested against passive Gaussian noise, never against worst-case attacks.

Our claim is that, in a two-path complex network, an asymmetric Hilbert-consistency target on attacked inputs recovers clean accuracy at least as well as AR-AT's feature matching. It should do so without Split-BatchNorm and with measurably less gradient conflict. We further claim the gain survives comparison against a real network with twice the width, but only when the network avoids the collapse identified by Real-Valued Backpropagation is Unsuitable for Complex-Valued Neural Networks. That paper proves that very wide complex networks with component-wise activations, trained by ordinary backpropagation, behave exactly like a real network with twice the units. Only "holistic" activations such as zReLU escape this. The collapse result gives the proposal a sharp prediction that could fail.

The Objective

Let $u(x)$ and $v(x)$ be the hidden features of the real-part and imaginary-part subnetworks for input $x$, and let $x'$ be an adversarial version of $x$. Let $\mathcal{H}$ be the discrete Hilbert transform, applied to the features as in the Analytic Neural Network, and let $\mathrm{sg}$ denote stop-gradient. We train with

$$ \mathcal{L} = \mathrm{CE}(x) + \mathrm{KL}\big(p(x)\,\|\,p(x')\big) + \lambda_1 \big\| v(x') - \mathcal{H}\,u(x') \big\|^2 + \lambda_2 \big\| h(u(x'),v(x')) - \mathrm{sg}[\,u(x),v(x)\,] \big\|^2 . $$

Here $\mathrm{CE}(x)$ is cross-entropy on the clean input, and the KL term is the divergence between the predicted class distributions $p(x)$ and $p(x')$ on clean and attacked inputs. The penalty weighted by $\lambda_1$ is the Hilbert-consistency term applied to the attacked input. The penalty weighted by $\lambda_2$ is AR-AT's one-way matching term, with $h$ its predictor head. Normalization is per-sample (LayerNorm or GroupNorm) rather than Split-BatchNorm.

The central hypothesis is that the $\lambda_1$ term alone, with $\lambda_2 = 0$, captures most of AR-AT's clean-accuracy benefit. Because the Hilbert target does not depend on the class, its gradient should be closer to orthogonal to the cross-entropy gradient than feature matching is. We will measure this directly, as the cosine similarity between the regularizer's gradient and the classification loss's gradient throughout training.

The Hilbert term is used only during training; inference is a single forward pass with no randomness. This leaves little room for gradient masking, the failure in which a defense merely hides useful gradients from the attacker rather than being genuinely robust. We still check for it, as described in the experiments.

Research Program

The program crosses three architecture families, all built at matched parameter counts:

  • a standard real network that reads the real and imaginary parts as two stacked input channels;
  • a Steinmetz two-path network with component-wise activations (applying a real activation separately to the real and imaginary parts);
  • the same two-path network with zReLU.

These are crossed with five training schemes: standard training, standard projected-gradient-descent adversarial training (PGD-AT), AR-AT, Hilbert-only regularization, and the combined objective above.

The width study sweeps hidden sizes over roughly an order of magnitude. Each two-path model is compared against a single real network with twice its hidden width, the exact equivalent class named by Tan et al.'s Theorem 7. That paper's theory predicts that the component-wise-activation two-path network's advantage over this real twin should shrink toward zero as width grows. It predicts that the zReLU network's advantage, if any, should persist.

We treat the collapse theory as a constraint, not an assumption. It is proven only in the infinite-width limit, and has been checked empirically only at widths of 100 to 1000 on MNIST. Whether it governs trade-offs under adversarial training at finite width is exactly what we test.

The threat model must respect complex geometry. An $\ell_2$ bound on a complex vector equals the $\ell_2$ bound on its real-imaginary concatenation, so we use that directly. An $\ell_\infty$ bound is different. We define it as a per-entry bound on the modulus $|\delta_k|$ of each complex perturbation entry $\delta_k$, rather than a separate bound on each real and imaginary component. We implement this projection inside APGD, the step-size-adaptive version of projected gradient descent.

Baselines

The naive reference points are standard training and standard PGD-AT on each architecture. They fix the size of the trade-off we are trying to shrink. For image-derived data, Phase-aware Adversarial Defense reports standard PGD-AT on ResNet-18 at 47.23% accuracy under PGD, against 58.24% for its own method. We cite these only as context, because our backbones differ.

The strongest existing method is AR-AT. It reports 90.89% clean accuracy and 50.77% accuracy under AutoAttack on CIFAR-10 with WideResNet-34-10, against 84.33% clean accuracy for TRADES at comparable robustness (Rethinking Invariance Regularization). Our backbones are smaller, so we do not claim to beat that absolute number. Instead we rerun AR-AT, including Split-BatchNorm and its automatic loss balancing, on every architecture in our study. Each rerun uses the same data, attack budget, number of training epochs, and a hyperparameter search of equal size (the same number of trials over $\lambda$ and learning rate). We report three seeds per configuration.

Two further baselines complete the set. The first is the obvious transplant: AR-AT placed unchanged on the Steinmetz two-path network. It isolates whether the Hilbert target adds anything beyond simply combining the two prior papers. The second is Phase-aware Adversarial Defense (PAD), the closest existing work linking phase and amplitude to robustness. PAD recombines the phase of an attacked image with the amplitude of a clean one, and at test time swaps amplitudes via AdaIN, a style-transfer method that renormalizes image statistics toward a reference. It then trains an ordinary real CNN on the results. We run PAD only on the image-derived dataset, since its style-transfer step presupposes images.

The key ablations are Hilbert-only versus matching-only versus both, one-way versus symmetric Hilbert penalty, and Split-BatchNorm versus per-sample normalization.

We will count the method as beating AR-AT if it meets all of the following conditions:

  • At matched AutoAttack robust accuracy, it achieves higher clean accuracy.
  • The clean-accuracy gap exceeds two seed standard deviations.
  • The result holds against the width-doubled real twin.
  • The method does not use Split-BatchNorm.

Why not just…?

Why not just stack real and imaginary parts as two channels and run AR-AT? This is the practitioner default and our main real-network comparison. The Steinmetz paper reports that such a network reaches 73.18% on CV-MNIST, the Fourier-transformed variant of MNIST, against 75.58% for the Analytic Neural Network. That gap was measured without any adversarial training, however, and says nothing about robustness. If the stacked network trained with AR-AT matches our method on the robust–clean trade-off, the proposal's architectural premise fails, and we will report it as such.

Why not just widen the real network? This is the most serious threat. Tan et al.'s theorem says a two-path network with component-wise activations is, in the wide limit, a real network with twice the units. Any gain we observe could therefore be capacity. The width-doubled twin and the width sweep are designed to settle this directly. If only the zReLU variant keeps its advantage at large width, the complex-structure explanation survives. If no variant keeps its advantage, the honest conclusion is that the decomposition helps only through extra capacity.

Why not just preprocess in the Fourier domain, as PAD does? PAD shows phase-level training helps against PGD, Carlini–Wagner, and spatial-transform attacks. It leaves the classifier real-valued, never measures what happens to clean accuracy, and depends on AdaIN style transfer, which introduces artifacts. Input filters in other settings also reject unusual benign inputs, as reported by Baseline Defenses for Adversarial Attacks Against Aligned Language Models. The PAD baseline measures directly whether our internal constraint beats external preprocessing on clean accuracy at equal robustness.

Why not just use a native complex network with complex backpropagation? Liouville's theorem rules out bounded activations that are complex-differentiable everywhere. The CVNN result of 71.72% on CV-MNIST reported in the Steinmetz paper trails both alternatives. We include the zReLU two-path network as the closest workable version of a native complex network.

Proposed Experiments

We use three benchmarks, each testing a different setting.

CV-CIFAR-10. This is CIFAR-10 with a fixed Fourier-transform front end, following the Steinmetz setup. Attacks perturb pixels with an $\ell_\infty$ budget of 8/255, so results sit in the standard setting evaluated by AutoAttack. No reported result was found on Lacuna for any adversarially trained Steinmetz or Analytic network here, nor for CV-CIFAR-10 clean accuracy.

CV-FSDD. This is the complex-transformed Free Spoken Digit Dataset, attacked with the modulus-bounded $\ell_\infty$ threat described above. No reported result was found on Lacuna for it.

RASPNet. RASPNet supplies genuine complex radar returns for target localization. Its hardest ("Eye-watering") Grand Canyon scenario reports localization errors of 180.14 m for the Adaptive Radar Transformer and 220.45 m for a regression CNN. RASPNet exceeds 16 TB, so we use a few scenarios per difficulty tier. Clean localization error is measured on the nominal terrain, and robust error is measured under bounded perturbations of the complex returns. This directly asks whether hardening destroys nominal accuracy, which no prior work has measured.

Robustness is always evaluated with the full AutoAttack ensemble from Reliable evaluation of adversarial robustness. That work found overestimated robustness in 48 of 49 models it tested. For the regression task we use APGD adapted to a regression loss plus Square Attack. We also add an adaptive attack that includes the Hilbert penalty in the attacker's loss.

The reported quantities are:

  • clean and robust Pareto curves, traced by sweeping $\lambda_1$;
  • confidence calibration, since RobustBench documents underconfidence in robust models;
  • the gradient-conflict cosine throughout training.

Three checks are decisive. First, Hilbert-only regularization should show lower gradient conflict than AR-AT's matching term, and at least equal clean accuracy at matched robustness. Second, the advantage should persist against the width-doubled real twin for zReLU but shrink for component-wise activations as width grows. Third, the black-box Square Attack should not find markedly lower robust accuracy than the gradient attacks; if it does, that signals gradient masking rather than genuine robustness. Failing the first check refutes the mechanism. Failing the second reduces the result to capacity.