Precision-Aware Regularization for Fixed-Point MIMO Precoding

When a base station with many antennas serves many users at once, it pre-distorts each user's signal so that the streams separate cleanly at the receivers. The standard way to do this, zero-forcing precoding, inverts the channel matrix. That inversion is fragile when users' channels point in similar directions, because the matrix becomes "ill-conditioned": some directions get multiplied by very large factors. Textbooks treat the damage as amplified thermal noise. Real precoders, however, run on fixed-point hardware with 8 to 16 bits per number, and rounding errors inside the inversion are amplified by the same large factors. This note proposes that the regularization knob already present in regularized zero-forcing (RZF) should be set by the hardware's word length as well as by the noise level. Choosing it this way should let a precoder drop several bits on crowded, line-of-sight channels without losing rate, and it predicts exactly which channels make floating-point simulations overstate hardware performance.

The simulation-to-silicon gap here is not a constant offset. It depends on the channel. Floating-point design studies report averages over channel draws. Fixed-point validation, such as the 0.15 dB implementation loss reported for an FPGA massive-MIMO precoder, is also usually done on benign channel ensembles. The failures sit in the tail where the matrix is nearly singular. We will build a closed-form predictor of fixed-point rate loss as a joint function of word length and condition number, and use it to choose the regularization parameter per channel realization. We will then check both the predictor and the precoder against bit-true simulation and an FPGA implementation, using realistic correlated mmWave channels.

Core Claim

Regularized zero-forcing uses the precoder $W = H^H(HH^H + \alpha I)^{-1}$. Here $H$ is the $K \times B$ downlink channel ($K$ users, $B$ antennas), and $\alpha \ge 0$ is the regularization parameter, conventionally set to $K\sigma^2/P$ with noise variance $\sigma^2$ and power budget $P$. A $b$-bit fixed-point inversion produces a relative error in $W$ of order $c\,\kappa_\alpha 2^{-b}$, where $c$ is an algorithm-dependent constant and $\kappa_\alpha$ is the condition number of the matrix being inverted. That condition number is:

$$\kappa_\alpha = \frac{s_{\max}^2 + \alpha}{s_{\min}^2 + \alpha}.$$

Here $s_{\max}$ and $s_{\min}$ are the largest and smallest singular values of $H$. The form of the error follows from standard first-order perturbation bounds for linear solves. The error appears at the users as extra inter-user leakage. Each user's effective signal-to-interference-plus-noise ratio therefore has three penalty terms: thermal noise, residual interference from regularization (which grows with $\alpha$), and quantization leakage of order $c^2\kappa_\alpha^2 2^{-2b}P$ (which shrinks with $\alpha$). The key coupling is that $\alpha$ suppresses noise enhancement and rounding amplification by the same mechanism. The best $\alpha$ therefore depends on $b$, and it rises sharply as $b$ falls on ill-conditioned channels.

The specific claims are these. First, a regularizer that accounts for word length recovers most of the rate lost to fixed-point arithmetic at low bit widths. Second, its gain concentrates in the high-$\kappa$ tail that average-case simulation hides. Third, the same model predicts the hardware rate loss well enough to size the datapath at design time.

Relation to Prior Work

The closest work is Finite Word Length Effects on Transmission Rate in Zero Forcing Linear Precoding for Multichannel DSL. It separates channel-estimation error from fixed-point rounding error in zero-forcing crosstalk cancellation and derives a closed-form bound showing that rate loss decays exponentially with word length. It also identifies 14 bits as sufficient for under 1% loss on loops longer than 300 m. Its proof requires the channel to be row-wise diagonally dominant, which holds for copper crosstalk but not for wireless multi-user channels. It also treats the precoder as fixed and asks only how many bits are needed. We drop the diagonal-dominance assumption, let the condition number enter explicitly, and turn the analysis into a control: the precoder changes $\alpha$ to fit the bits available. This paper supplies the method template (error decomposition followed by a closed-form rate-loss bound), and we reuse it.

Two hardware-oriented precoding papers constrain the design space. 1-bit Massive MU-MIMO Precoding in VLSI builds C1PO and an inversion-free alternative, C2PO, on a Virtex-7 FPGA, and reports fixed-point loss within 0.15 dB of infinite precision. It avoids inversion through 1-bit nonlinear precoding rather than studying linear precoding when inversion is required, and it evaluates flat-fading channels. Hardware-Aware Beamspace Precoding for All-Digital mmWave Massive MU-MIMO sparsifies the precoder in a spatial-Fourier "beamspace" to cut multipliers, reaching up to 2.91x speedup over maximum-ratio transmission on QuaDRiGa mmMAGIC channels. It gives no fixed-point or RTL results and leaves open what happens when the selected beam submatrix is itself ill-conditioned. Both papers motivate the hardware angle. Neither ties precision to channel conditioning.

On the wireless side, Tajer and Nosratinia and Tajer, Nosratinia and Al-Dhahir prove that zero-forcing equalizers are stuck at diversity order one, while MMSE escapes by trading interference for bounded noise. Our claim extends this argument to rounding noise: the same trade should absorb quantization error. Nguyen et al. show that optimizing $\alpha$ jointly with power allocation instead of using the heuristic value raises supported users from 12 to 18 at 0.8 bps/Hz with 16 antennas. Their result motivates the view that $\alpha$ is worth optimizing, but their optimization assumes exact arithmetic. Work on channel reshaping (PR-MIMO) and integer-forcing receivers (Zhan et al.) attacks ill-conditioning at the antenna or coding layer and is complementary.

The simulation-to-silicon literature supplies the methodological stance. Air Learning closes a 37.73% flight-time gap to 0.5% by profiling the target hardware and injecting its behavior into simulation. Rasch et al. show that output-side noise in analog accelerators hurts more than weight noise, and that injecting hardware noise during design recovers accuracy. Our analogue is to put a calibrated model of datapath rounding into the precoder's own design rule rather than validating against hardware only afterwards.

Research Program

The first component is analysis. We will derive the rate-loss bound for fixed-point RZF without diagonal dominance, for the two inversion schemes most used in silicon: Cholesky-based solves and a fixed-iteration Neumann or Gauss–Seidel approximation. The output is an explicit expression in $b$, $\kappa_\alpha$, $K$, and the noise level, with the constant $c$ calibrated once per arithmetic scheme from bit-true runs. From the bound we get a closed-form or one-dimensional-search rule for $\alpha^*(b, \kappa, \sigma^2)$. A cheap estimate of $\kappa$ can come from the diagonal of the Cholesky factor, which the hardware computes anyway.

The second component is implementation. We will write a bit-true C++ model of RZF with configurable word length, rounding mode, and saturation. We will also build an FPGA implementation on the Virtex-7 class of device used by Castañeda et al., parameterized in word length. The FPGA serves two purposes. It confirms that the bit-true model captures what the hardware actually does, including DSP-slice rounding and overflow handling. It also gives measured area, power, and throughput per bit-width, so that bit savings convert into hardware cost.

The third component is a gap audit. For each channel ensemble we will report the difference between floating-point and fixed-point rate as a function of channel condition number, not only as an average. This shows where design-time simulation misleads.

Baselines

The naive reference is classical zero-forcing ($\alpha = 0$) at each word length. It shows the raw size of the problem and should collapse first on correlated channels.

The main competitor is conventional RZF with $\alpha = K\sigma^2/P$, computed in fixed point at the same word length. This is what designers actually deploy, and it already bounds the condition number, so it is the method that could make our idea unnecessary.

The strongest existing method is the jointly optimized RZF of Nguyen et al., optimized in floating point and then executed in fixed point. Its reported result is 18 supported users at 0.8 bps/Hz with 16 base-station antennas in their hexagonal single-cell model, against 12 for heuristic RZF. We will reproduce that floating-point figure before quantizing, so the comparison is anchored.

For hardware precision, the reference figure is the 0.15 dB fixed-point loss reported by Castañeda et al.. For datapath sizing it is the 14-bit, under-1% result of Sayag et al..

All methods will see identical channel draws, identical channel estimates, and identical fixed-point arithmetic. Any per-scenario tuning given to our rule, such as calibrating $c$, will also be granted to the baselines as a grid search over a fixed scalar multiplier on their $\alpha$.

An oracle baseline searches $\alpha$ per channel draw directly in bit-true simulation. It bounds what any regularization rule can achieve.

The ablation that isolates the key ingredient is a precision-aware rule with the condition number removed. That is, $\alpha$ is inflated by a single word-length-dependent constant, not by a per-channel $\kappa$-dependent amount.

Our method beats the competition if it matches conventional RZF's sum rate and fifth-percentile user rate with at least two fewer bits on the correlated ensembles, comes within a small fraction of the oracle, and clearly outperforms the condition-number-free ablation in the high-$\kappa$ tail.

Why not just…?

Why not just use standard MMSE regularization, which already bounds the condition number? This is the most serious threat. At high SNR the conventional $\alpha$ is small, so $\kappa_\alpha$ stays large exactly where a linear precoder is expected to deliver high spectral efficiency. The diversity results of Tajer et al. show that noise-only regularization is calibrated to thermal noise, not to rounding. If our experiments show that conventional RZF already makes quantization leakage negligible at every word length of practical interest, down to about 8 bits, on QuaDRiGa line-of-sight channels, the proposal fails and we will report that. The model predicts the opposite at high SNR with 8 to 12 bits.

Why not just widen the datapath? Widening always works numerically. The question is what it costs at hundreds of antennas, where Castañeda et al. already find interconnect and power to be binding constraints. We will treat "add bits until conventional RZF matches floating point" as a baseline and report its measured FPGA area and power. Our method has to buy the same rate more cheaply. If the saving turns out to be only one bit, widening is the better engineering choice, and we will say so.

Why not just run bit-true simulations instead of building a model? Bit-true simulation is our ground truth for arithmetic error, and the FPGA may add little beyond it on that axis. Its main role is to supply measured cost. The analytical model is needed for a different job: choosing $\alpha$ online for each channel, which offline simulation cannot do. The oracle baseline measures how much is lost by using the model instead of exhaustive search.

Why not just avoid inversion with C2PO or maximum-ratio transmission? Those methods target a different regime. C2PO is built for 1-bit converters, and maximum-ratio transmission leaves strong inter-user interference, as the beamspace comparisons in Gönültas et al. illustrate. Both are poor fits for multi-level modulation such as 64-QAM, where linear inversion remains standard. We include maximum-ratio transmission only as a reference.

Proposed Experiments

The primary benchmark is the QuaDRiGa mmMAGIC urban-microcell model at 60 GHz with 128 antennas and 16 users, as in Gönültas et al., split into line-of-sight and non-line-of-sight. We will also apply the dense precoders to the beamspace submatrices selected by their sparse methods, to test their open question about ill-conditioned active beams.

The second benchmark is the hexagonal single-cell model with exponential antenna correlation from Nguyen et al.. Sweeping the correlation coefficient and the user count up to the antenna count gives direct control over conditioning.

As a sanity check, we will apply our bound in the diagonally dominant regime using the Werner copper model. It should recover the roughly 14-bit requirement of Sayag et al.. If it does not, the derivation is wrong.

Metrics are sum rate, fifth-percentile user rate, uncoded error rate for 64-QAM, and supported users at 0.8 bps/Hz, reported against word lengths from 6 to 16 bits. Hardware metrics are FPGA area in LUTs and DSP slices, power, and throughput.

Three checks are decisive. First, the predictor's estimate of fixed-point rate loss should track bit-true and FPGA measurements across condition-number bins, including the top decile, rather than only on average. Second, at matched rate the precision-aware rule should save at least two bits over conventional RZF on correlated channels, and this saving should shrink toward zero on well-conditioned ensembles; a saving that does not shrink would suggest it comes from something other than conditioning. Third, removing the condition-number term should erase most of the tail gain. If the gain survives without that term, the mechanism is generic regularization inflation, and the claim reduces to a simpler, still useful, design rule.