Quadratic Neurons for Learning to Beamform from Raw Pilots

Many wireless control problems are now solved by "learning to optimize": a neural network reads the measured radio environment and outputs a control decision, such as the phase setting of each element of a reflecting surface, and it is trained with no labels by maximizing the physical objective (for example, the total data rate) directly through backpropagation. The physics of these objectives is second-order. Received power is the squared magnitude of a channel–beamformer product, and interference is a sum of such squares. Yet the networks used are stacks of ordinary linear-plus-ReLU layers, which cannot multiply two inputs and must spend depth to approximate products. A quadratic neuron is a unit that multiplies two learned projections of its input, plus an optional learned weighting of squared inputs. It computes such products in one step.

This note proposes to test a specific, falsifiable claim. When the network must map raw received pilot signals to beamformers, the decision depends on second-order statistics of those pilots. Putting quadratic neurons in the first layer should then let a shallow network close the gap to perfect-channel performance with fewer pilots and lower latency than a conventional network of matched compute. When the inputs are already squared channel gains, as in classic power allocation, the advantage should vanish. The interesting result is the contrast between these two regimes, not "quadratic is better" in general. The contrast also makes the claim checkable, because it predicts both a positive and a null result.

Core Claim and What Is New

Two lines of work supply the ingredients and motivate the gap. On the optimization side, Learning to Beamform for Intelligent Reflecting Surface with Implicit Channel Estimate trains a fully connected network on raw pilots with negative sum-rate as the loss, skipping explicit channel estimation. Unsupervised Learning for Passive Beamforming does the same for received signal-to-noise ratio, using an output layer that enforces unit-modulus phases. Both use conventional neurons. On the architecture side, One Neuron Saved is One Neuron Earned proves that a product costs a constant number of quadratic neurons but a number of ReLU neurons that grows logarithmically in the target error. On Expressivity and Trainability of Quadratic Networks shows that such networks are unstable unless their quadratic weights start at zero and are trained gently, a recipe the authors call ReLinear. That training evidence comes from vision tasks with standard losses.

The closest work joining the two directions differs from this proposal in specific ways. The quadratic heterogeneous autoencoder of Quadratic Neuron-empowered Heterogeneous Autoencoder for Unsupervised Anomaly Detection is unsupervised but minimizes reconstruction error, with no external physical objective and no hardware constraint. The Brain Quadratic Network of Do We Really Need Message Passing in Brain Network Modeling? replaces message passing with elementwise products on dense correlation graphs, but it is trained with supervised labels. This proposal is the first to put quadratic neurons inside a label-free, physics-defined objective. It asks a structural question that neither paper can ask: does the benefit track whether the second-order structure is already present in the input?

The unit we use, following Fan et al., is

$$y = \sigma\big((w_1^\top x + b_1)(w_2^\top x + b_2) + w_3^\top (x \odot x) + b_3\big),$$

where $x$ is the stacked real and imaginary parts of the received pilots, $w_1, w_2, w_3$ are weight vectors, $b_1, b_2, b_3$ are scalar biases, $\odot$ is elementwise product, and $\sigma$ is an activation. The term $x \odot x$ gives per-entry energy directly. The bilinear term gives one learned rank-one quadratic form per neuron. We also test a complex-valued variant that computes $\mathrm{Re}(u^H x \, x^H v)$ on the complex pilots, where $u$ and $v$ are complex weight vectors. This form is invariant to a common phase rotation of the channel, as the sum-rate itself is. The network is trained by minimizing $\mathcal{L}(\theta) = -\mathbb{E}\left[\sum_k \log_2(1+\mathrm{SINR}_k(\theta))\right]$, where $\theta$ are the network weights and $\mathrm{SINR}_k$ is user $k$'s signal-to-interference-plus-noise ratio under the network's output beamformers and phases.

Research Program

The program has three parts. First, we rebuild the implicit-estimation setting of Jiang, Cheng and Yu: an IRS-aided multi-user downlink (IRS meaning intelligent reflecting surface, a panel of passive reflectors with adjustable phases), with pilot length swept from very short up to the roughly 180 pilots the classical estimate-then-optimize pipeline needed. The single-antenna and multi-antenna passive beamforming settings of Gao et al. serve as a second regime. In both, we replace only the first one or two layers with quadratic neurons, keeping their output layers and constraint handling.

Second, we treat trainability as a first-class question rather than an afterthought. Sum-rate landscapes are full of plateaus and saddles, and quadratic units amplify input scale. We therefore compare three training schemes across many seeds: ReLinear with gradient shrinking, ReLinear with weight shrinking, and the reduced learning rate for quadratic weights used in the heterogeneous autoencoder paper. We report divergence rates and the spread of final sum-rate, not just the best run.

Third, we run the negative control. On the single-hop power allocation benchmark of Efficient power allocation using graph neural networks and deep algorithm unfolding, the inputs are already squared channel gains and the output is a power vector. There we insert quadratic neurons into the same graph network and predict no meaningful gain. If they help just as much there, the "matching the physics" explanation is wrong and the gain is generic capacity.

Baselines

The naive reference is the classical pipeline Jiang et al. compare against: linear minimum-mean-square-error channel estimation followed by block coordinate descent on the estimated channel. Next to it sits the perfect-channel upper bound computed the same way. Every learned method is reported as a fraction of that upper bound at each pilot length.

The strongest existing method is the conventional fully connected network of Learning to Beamform for Intelligent Reflecting Surface with Implicit Channel Estimate. That paper reports that with 54 pilots it reaches a sum rate within 6.7% of perfect channel knowledge, where the classical pipeline needed roughly 180 pilots. We reimplement it in our simulator and confirm we reproduce a gap near 6.7% at 54 pilots before comparing. It then gets the same hyperparameter search budget as our model, measured in trials and GPU-hours, including a free choice of depth and width. Beating it means one of two things: reaching its 54-pilot gap with clearly fewer pilots, or reaching a clearly smaller gap at 54 pilots, at matched floating-point operations per inference, with the difference holding across seeds. For passive beamforming, the reference is RISBFNN from Unsupervised Learning for Passive Beamforming, which reports 88% to 96% of semidefinite-relaxation performance in multi-antenna setups at 0.04–0.11 ms. Beating it means a higher fraction at equal or lower measured latency. For the power allocation control, the reference is unfolded WMMSE, which that paper reports at a mean sum-rate of 83.21 against 82.94 for classical WMMSE. There we expect a tie.

Three further baselines isolate the key ingredient, and all are matched in compute. A gated linear unit layer, meaning the product of one linear projection and a sigmoid of another, tests whether any multiplicative gate suffices. A plain bilinear layer without the $x \odot x$ term tests whether the explicit energy term matters. An explicit pilot outer-product input feeds the conventional network the full second-order features, testing whether learning the products matters at all. We also ablate quadratic neurons in the first layer only versus throughout, and with versus without ReLinear.

Why not just…?

Why not just feed the conventional network the pilot outer product as input? This is the most direct alternative, and in small systems it may simply work. Its cost is that the full second-order feature set grows with the square of the pilot dimension. Pilot dimension scales with surface size and pilot length, so the input layer becomes enormous exactly where IRS systems are heading. Quadratic neurons learn a small number of task-relevant rank-one forms at linear cost per neuron. The outer-product baseline at matched compute settles this. If it matches us at every surface size we test, we will report that the benefit is the features, not the neuron.

Why not just make the ReLU network deeper or wider? Fan et al. show that product approximation costs ReLU networks a number of parameters growing with $\log(1/\epsilon)$. Extra depth also adds serial latency, which competes with coherence-time budgets as tight as the 13.77 ms indoor estimate in Gao et al. Our baseline gets free depth and width under an equal tuning budget. The comparison is therefore made at matched operations and measured batch-one latency, not at matched parameter count, which would flatter quadratic units.

Why not just use gated linear units, which already multiply two projections? This is a real threat, because a gated unit with a near-linear gate spans much of the same bilinear space. We do not argue around it. The gated-unit baseline is run alongside, and the falsifiable difference is the explicit energy term and the phase-invariant complex form. If gated units match the quadratic neuron, the honest conclusion is that multiplicative interaction in the first layer is what matters. That would still be the first such finding for label-free wireless optimization, and the pilot-versus-gain contrast would still test it.

Why not just unfold a classical algorithm, which already contains the right products? Unfolding WMMSE works when channels are known. In the implicit setting there is no channel estimate to unfold over, which is why Jiang et al. use a generic network. We include unfolding only in the power-allocation control, where we expect quadratic neurons to add nothing.

Decisive Checks

The idea succeeds if three things hold. Quadratic first layers reach the reproduced 54-pilot gap with substantially fewer pilots than the tuned conventional network at matched compute. That advantage grows with the number of surface elements. And it disappears in the power-allocation control. The idea fails, or is reduced to "multiplicative gates help," if the gated-unit or outer-product baselines match it, or if the gain is as large on squared-gain inputs. The trainability study reports, as a standalone result, whether ReLinear keeps divergence rates under sum-rate losses comparable to those of conventional networks. Wall-clock latency on a CPU and a GPU at batch size one is reported alongside floating-point operations, so that the parameter savings are not mistaken for speed.