Matching the Operating Point: Cheap Hyperparameter Search for Learned Covariance Activity Detectors

When millions of sensors share a cellular base station, most of them are silent at any given moment. The receiver has to work out which few are transmitting from a short burst of overlapping signals. One strong way to do this skips reconstructing each device's channel. Instead, the receiver looks only at the sample covariance of what its many antennas hear: the $L \times L$ matrix of how the received pilot symbols co-vary. Each active device leaves a fingerprint in that matrix. This covariance-based approach can detect roughly $L^2$ active devices with pilots of length $L$, whereas channel-reconstruction methods stop near $L$. The catch is speed. The maximum-likelihood solvers that read the fingerprints are iterative and depend on hand-set thresholds and step sizes. Even accelerated versions take seconds on ten thousand devices.

Making such a solver learned and fast is natural: "unfold" a fixed number of its iterations into network layers and train the thresholds. Doing so, however, creates a hyperparameter search whose every trial means training at full system scale. This note proposes a specific way to make that search cheap. Hyperparameters are tuned on small wireless systems whose dimensions are chosen to sit at the same operating point as the target system. "Operating point" here means the same load relative to the covariance scaling law, the same pilot-to-antenna ratio, and the same detection difficulty for a reference solver. The best configurations found there are then transferred to full scale. The central bet is that hyperparameters transfer across system sizes only when difficulty is held fixed. Naively shrinking the device pool or antenna count, as resolution-shrinking does for images, should scramble the ranking of configurations.

Core Claim

The idea borrows from Speeding up the Hyperparameter Optimization of Deep Convolutional Neural Networks. That work searched convolutional-network hyperparameters on downscaled images and transferred them to full resolution, cutting search time by 9% to 42%. Its premise was that hyperparameter importance is resolution-invariant. The research file flags exactly why that premise is dangerous here. Detection quality in covariance processing depends on how close the sample covariance is to its expectation, and that is governed by the ratio of pilot length $L$ to antenna count $M$. It also depends on how heavily the system is loaded relative to the scaling law $K = O(L^2/\log^2(BN/L^2))$ proved in Covariance-Based Activity Detection in Cooperative Multi-Cell Massive MIMO: Scaling Law and Efficient Algorithms. Here $K$ is the number of active devices, $N$ the device pool, and $B$ the number of cooperating cells. A small system with $M=16$ is not a blurry version of $M=256$. It is a different statistical regime.

A second observation decides which dimension to shrink. Once the covariance $\hat{\Sigma}_y = \frac{1}{M}YY^H$ is formed, the detector's cost depends on $N$ and $L$, not on $M$. Shrinking antennas therefore saves almost nothing. The real cost drivers are the device pool and pilot length. We claim that rank correlation between small-system and full-system validation performance stays high only when the fidelity is built by shrinking $(L, N, K, M)$ jointly while holding three quantities fixed: the activity ratio $K/N$, the aspect ratio $L/M$, and the miss rate that a reference maximum-likelihood solver achieves at a fixed false-alarm rate. Shrinking any single dimension should cause rank inversion. Rank inversion means a configuration that loses at small scale wins at full scale, which is the known failure mode of multi-fidelity search.

Research Program

The learned detector comes first, because without it there is nothing expensive to tune. We unfold the active-set coordinate descent of Accelerating Coordinate Descent via Active Set Selection for Device Activity Detection for Multi-Cell Massive Random Access and the inexact variant of Covariance-Based Activity Detection in Cooperative Multi-Cell Massive MIMO into $T$ layers. Each layer runs four steps. It computes the gradient of the covariance log-likelihood with respect to the per-device power vector $p$. It selects the devices whose optimality conditions are most violated. It applies a coordinate or projected-gradient update with a learned step. It then passes $p$ through a learned soft threshold. The soft threshold is a trainable sigmoid, borrowed from the soft activity module in Deep-Unfolding-Based Joint Activity and Data Detection for Grant-Free Transmission in Cell-Free Communication Systems (DU-JAD). That paper unfolds forward-backward splitting on the raw received signal and showed 10 learned layers can match 200 classical iterations. We reuse its unfolding recipe but feed the layers the covariance rather than the raw signal, which is what preserves the $L^2$ capacity.

The input is always the PSD covariance matrix, entering only through likelihood gradients. As a result, the learned layers never need to convolve over the matrix and cannot break its structure. This is a deliberate answer to the concern, raised in the file, that grid convolutions mishandle Hermitian matrices. Because the unknown is $p_n = a_n\gamma_n$, the network also estimates each device's large-scale fading $\gamma_n$ jointly instead of assuming it known.

Unfolding turns the solver's inner hyperparameters into trained weights: thresholds per layer, step sizes per layer, and active-set sizes. That leaves an outer set of hyperparameters, each requiring a full training run. These are depth $T$, active-set fraction, learning rate and schedule, weight decay, the loss weighting between miss and false-alarm, and the training signal-to-noise range. This outer set is what we search.

The second component is the fidelity construction. For a target system $(L, N, K, M, B)$ we build a ladder of smaller systems by two rules. The scaling-law rule holds fixed the normalized load

$$\rho = \frac{K \log^2(e + BN/L^2)}{L^2},$$

where the $e$ inside the logarithm only keeps the expression positive when $N < L^2$. The same rule also fixes $K/N$ and $L/M$. The error-matched rule instead searches, among small systems with fixed $K/N$ and $L/M$, for the one whose reference solver has the same miss probability at false-alarm $10^{-3}$ as the target. The reference solver is untuned active-set coordinate descent. Running it costs seconds at small scale and one run at full scale.

We plug both ladders into BOHB from BOHB: Robust and Efficient Hyperparameter Optimization at Scale. BOHB is successive halving guided by kernel density models, and here system size replaces epochs as the budget axis. The analysis we care most about is rank agreement. For a fixed pool of 200 random configurations, each trained at every rung, we measure Spearman correlation between rung and full-system validation performance. We also apply functional analysis of variance to see whether hyperparameter importance is preserved, mirroring the analysis in the image-resolution paper.

Baselines

The do-nothing reference is classical active-set projected gradient with Barzilai-Borwein steps, from An Efficient Active Set Algorithm for Covariance Based Joint Data and Activity Detection for Massive Random Access with Massive MIMO, run with its published default thresholds. That paper reports about 4 seconds of CPU time at $N=10{,}000$, $M=256$, $L=150$, compared with over 10 seconds for random coordinate descent. For multi-cell runs the reference is active-set coordinate descent, reported at about 12 seconds versus 27 seconds at 1,000 devices per cell, with miss/false-alarm curves overlapping those of the exact solver. No reported numeric miss or false-alarm rate was found on Lacuna for any covariance method, so we will regenerate those curves ourselves. A learned detector beats this reference if it matches the reference's miss rate at false-alarm $10^{-3}$ with at least an order of magnitude less wall-clock time on identical hardware, or lowers the miss rate at equal time.

The strongest classical competitor is the same solver with its thresholds, decay rates and tolerances tuned by BOHB at full scale. Its tuning budget is the same compute our method spends on search plus training. This matters because a well-tuned classical solver may close most of the gap that learning promises.

The strongest learned competitor is DU-JAD, retrained on our synthetic channels with the same training set and tuning budget. It reports a user detection error rate of 13.5% of its best baseline's at 10 iterations with 60 access points and 400 users. We expect it to degrade once $K$ exceeds $L$. If it does not, the covariance premise is wrong for this problem size.

On the search side, the strongest competitor is BOHB with the usual epoch-count fidelity at full system size, under the same total GPU-hour budget. There are two ablations. The first is naive shrinking: reduce $N$ alone, or $M$ alone, keeping everything else fixed. It isolates whether operating-point matching, rather than simply using smaller systems, carries the gain. The second is random search at full scale, which sets the floor.

Our search wins if it reaches a configuration within 2% relative miss rate of the best full-scale configuration using at most a third of the GPU-hours that epoch-fidelity BOHB needs.

Why not just…?

Why not just tune the classical solver and skip learning? This is a real threat. Classical active-set solvers already lose no accuracy relative to exact maximum likelihood. Tuning their few thresholds is cheap because no training is involved. The tuned-classical baseline settles this directly. If the unfolded detector cannot beat it at matched latency, the learned component is unjustified. The proposal then collapses to the fidelity study, applied to the classical solver's own thresholds, which remains useful for retuning across deployments.

Why not just use BOHB with epochs as the cheap proxy? Epoch budgets are standard and were behind BOHB's reported speedup of up to 100 times over Hyperband on the poker dataset. Epoch-based pruning still pays full-system cost per step, though, and it is exposed to early-epoch rank inversion. Our search cuts the cost per step instead. The head-to-head under equal GPU-hours decides it. If epoch-BOHB matches us, our contribution reduces to the rank-correlation diagnostic, and we will say so.

Why not just shrink the system the way image-resolution search shrinks images? The covariance scaling law and the $L/M$ dependence of sample-covariance error predict that naive shrinking moves the problem into a different regime. The image result does not carry over automatically. The naive-shrink ablation is designed to show this through measured rank correlations. If naive shrinking transfers as well as matched shrinking, our central claim is false.

Why not just learn hyperparameters online with Self-Tuning Networks? Self-Tuning Networks tune hyperparameters inside one training run by gating hidden units with a learned function of the hyperparameters. They beat Bayesian optimization on CIFAR-10 validation loss, 0.575 versus 0.636. In an unfolded solver, however, the hyperparameters that most benefit from schedules (thresholds and steps) are already per-layer trained weights, so an online schedule adds little. Self-Tuning Networks also double layer parameters and do not handle discrete choices like depth well. We include them as an optional comparison on the continuous outer hyperparameters only.

Proposed Experiments

All data are synthetic, following the cited setups. The single-cell setting is $M=256$, $L=150$, $N$ from 1,000 to 10,000, with 10% activity. The multi-cell setting is the hexagonal 7-cell layout with 1,000 devices per cell, using both discrete and continuous signature sequences, and adds unknown large-scale fading and an SNR sweep. The metrics are miss probability at false-alarm $10^{-3}$, wall-clock latency, GPU-hours spent searching, simple regret against the best full-scale configuration, and Spearman rank correlation between each fidelity rung and full scale.

Three checks decide the project. First, matched ladders must keep rank correlation clearly above naive ladders at the smallest rung. If both fall below roughly 0.5, the transfer hypothesis fails. Second, the searched unfolded detector must beat tuned classical active-set descent at equal latency. Third, the unfolded covariance detector must keep working past $K = L$, where DU-JAD should break down. A final transfer test checks that hyperparameters chosen at one target operating point remain near-optimal when only SNR shifts. This tells us whether operating-point matching captures the right notion of difficulty or only one slice of it.