Abstract
Millimeter-wave (mmWave) MIMO systems are essential for achieving massive bandwidth in next-generation wireless communications. However, the high power consumption and hardware costs associated with fully-digital transceivers remain significant barriers. This paper proposes a novel hybrid analog-digital transceiver design that incorporates Tomlinson-Harashima Precoding (THP), a nonlinear technique, to suppress multiuser interference. By jointly optimizing digital precoding and analog phase shifters through iterative and two-timescale algorithms, the proposed system approaches the performance of fully-digital architectures while maintaining low hardware complexity and reducing channel state information (CSI) feedback overhead.
Introduction
In massive MIMO mmWave systems, the physical feasibility of packing hundreds of antennas at a base station is offset by the prohibitive cost and power requirements of providing a dedicated Radio Frequency (RF) chain for every antenna. Hybrid analog-digital (A/D) architectures address this by using a limited number of RF chains connected to a larger antenna array via analog phase shifters.
While hybrid designs are cost-effective, most existing solutions rely on linear precoding, which often fails to adequately manage multiuser interference, especially when the number of RF chains is restricted. Nonlinear precoding, specifically Tomlinson-Harashima Precoding (THP), can pre-cancel known interference at the transmitter using modulo arithmetic. However, optimizing THP within a hybrid framework is challenging due to the non-convex "unit modulus" constraints of analog phase shifters and the high latency associated with real-time CSI feedback. This research addresses these challenges by developing robust optimization algorithms for nonlinear hybrid transceivers.
Methodology
The core of the proposed system is a joint optimization of the digital THP matrices and the analog beamforming matrices to minimize the system's Mean Square Error (MSE), subject to transmit power and hardware constraints.
Optimization Frameworks
Two primary algorithms were developed to handle the optimization:
- Single-Timescale Joint Design (Algorithm 1): This approach utilizes a Block Coordinate Descent (BCD) algorithm to transform the coupled, non-convex problem into tractable sub-problems. It iteratively solves for digital precoding, feedback matrices, and analog beamforming in closed forms until convergence.
- Two-Timescale Joint Design (Algorithm 2): To mitigate CSI feedback overhead, the authors proposed a framework using Two-Stage Online Successive Convex Approximation (TOSCA). In this model, analog phase shifters are updated slowly based on long-term statistical channel data (long-timescale), while digital matrices are updated rapidly based on low-dimensional instantaneous CSI (short-timescale).
Cancellation Ordering
In THP, the sequence in which user interference is cancelled significantly affects performance. Since calculating the exact MSE for all possible permutations is computationally expensive, the authors derived a mathematical lower bound for the MSE to determine a near-optimal cancellation order with low complexity.
Experimental Setup
The algorithms were evaluated using simulated narrow-band mmWave channel models featuring a uniform linear antenna array. The simulation environment typically involved a base station with 32 antennas and 8 RF chains serving 4 users.
Key Insights and Results
The research yielded several critical findings regarding the efficacy of nonlinear hybrid designs:
- Performance Gains: The BCD-based nonlinear joint design significantly outperforms traditional linear hybrid designs. It effectively "pre-subtracts" multiuser interference, allowing the hardware-constrained hybrid system to nearly match the performance of a theoretical fully-digital transceiver.
- The Importance of Ordering: The proposed cancellation ordering scheme provided a substantial ~3dB gain at a Symbol Error Rate (SER) of $2 \times 10^{-7}$ compared to random cancellation ordering, highlighting that intelligent sorting is vital for THP effectiveness.
- Robustness to Latency: A major advantage of the two-timescale approach is its resilience to feedback delays. In simulations with a 5ms CSI feedback delay, single-timescale algorithms suffered from high error rates, whereas the two-timescale algorithm maintained stable and excellent SER performance.
Conclusion
This work demonstrates that nonlinear hybrid transceivers using THP can bridge the performance gap between low-cost hybrid architectures and high-performance fully-digital systems. By splitting optimization across two timescales, the design also solves the practical issue of high feedback overhead in massive MIMO networks. Future research directions include extending this nonlinear framework to wideband scenarios, such as OFDM-based 5G/6G systems, to account for frequency-selective fading.
Summary updated