Recurrent Transformer-Based Near-and Far-Field THz Wideband Channel Estimation for UM-MIMO

Dmitry Artemasov, Alexander Shmatok, Kirill Andreev, Alexey A. Frolov, Manjesh Kumar Hanawal, Nikola Zlatanov

TL;DR

This paper introduces a novel deep learning approach, the Block Recurrent Transformer (BRT), for estimating wireless channels in the challenging Terahertz (THz) spectrum for Ultra-Massive MIMO (UM-MIMO) systems. The BRT model significantly improves channel estimation accuracy and generalization capabilities, outperforming existing state-of-the-art methods, particularly in complex hybrid near- and far-field scenarios.

What Problem Does This Paper Address?

The advent of 6G wireless networks is pushing towards the Terahertz (THz) frequency band to achieve unprecedented data rates. THz communications, combined with Ultra-Massive MIMO (UM-MIMO) systems, offer a promising path. However, these systems introduce significant challenges for channel estimation. Firstly, the large antenna apertures and high frequencies mean that users can be in either the near-field or far-field region, or a combination of both (hybrid-field), making traditional channel estimation techniques insufficient. Secondly, the limited number of radio frequency (RF) chains in hybrid beamforming architectures further complicates the estimation process. Existing methods often struggle with the complex interplay of near- and far-field effects, varying propagation paths, and wideband operation, leading to performance degradation. This paper aims to bridge this gap by developing a robust and accurate channel estimation method for these demanding THz UM-MIMO scenarios.

Methods

The core of this paper is the Block Recurrent Transformer (BRT) architecture, adapted for channel estimation. Unlike standard transformers that process sequences in parallel, the BRT introduces recurrence, allowing it to iteratively refine channel estimates. Key aspects of the BRT approach include:

  • Block Recurrent Transformer (BRT) Cell: This is the fundamental building block, integrating both vertical (layer-wise) and horizontal (time-step) processing. It uses attention mechanisms to capture contextual information and a gating mechanism to manage the flow of information across time steps, enabling it to learn from past estimations.
  • Iterative Refinement: The BRT model is applied recurrently over multiple time steps. At each step, it takes the previous channel estimate and memory states to produce a refined estimate. This iterative process allows the model to progressively denoise and enhance the channel estimation.
  • Constant Parameter Training: A single transformer block with state memory is trained and then iteratively applied. This ensures that the number of trainable parameters remains constant, regardless of the number of refinement iterations, making the model efficient.
  • Wideband Channel Estimation: The BRT architecture is extended to handle wideband signals by treating OFDM subcarriers as sequential tokens. This allows the model to capture spectral dependencies between subcarriers using multi-head self-attention and recurrence.
  • Online Dataset Generation: To combat overfitting and improve generalization, the training data (channel responses and measurements) is generated stochastically on-the-fly for each training epoch. This ensures a diverse and representative dataset.

Datasets

The paper does not explicitly mention pre-existing, named datasets for training. Instead, it relies on synthetically generated data for training and validation. This data is generated using a stochastic model that simulates the hybrid-field THz UM-MIMO channel, with parameters outlined in Table I. The generation process is done "on-the-fly" during training, meaning new data is created for each epoch, which is a key strategy to prevent overfitting and enhance generalization.

Key Insights and Ideas

  • Recurrence is Key for Iterative Refinement: The BRT's recurrent nature is crucial for its ability to iteratively improve channel estimates. This is a departure from traditional parallel processing in transformers and proves highly effective for tasks requiring progressive refinement.
  • Generalization Through Online Data Generation: The strategy of generating training data dynamically for each epoch is a powerful technique for improving the model's robustness and its ability to generalize across various channel conditions (e.g., different scatterer distances, number of paths, and bandwidths).
  • Hybrid-Field Challenges are Significant: The paper highlights that the combination of near- and far-field effects, coupled with wideband operation and hardware constraints in THz UM-MIMO, creates a highly ill-conditioned channel estimation problem, making it a stringent testbed for new algorithms.
  • Transformer Architecture Adapts Well to Wireless: The success of transformers in natural language processing and computer vision is now being effectively leveraged in wireless communications, with the BRT showing particular promise for channel estimation.

Key Results

  • Significant NMSE Improvement: The proposed BRT model achieves approximately 5 dB and 7.5 dB improvement in normalized mean squared error (NMSE) over state-of-the-art solutions in narrowband and wideband scenarios, respectively.
  • Superior Performance: The BRT model consistently outperforms existing classical and learning-based baselines, including FPN-OAMP (a current state-of-the-art DNN-based method), across various SNR ranges.
  • Robust Generalization: The model demonstrates strong adaptability to varying scatterer distributions, numbers of propagation paths, and wideband conditions, even when trained on a single baseline channel.
  • Efficient Inference: While training is computationally intensive, the BRT model exhibits a favorable trade-off between accuracy and computational cost during inference, with a lower per-sample inference time compared to FPN-OAMP, especially at larger batch sizes and higher subcarrier configurations.

Limitations and Future Work

  • Computational Cost of Training: The paper acknowledges that training the BRT model requires significant computational resources and time, despite the efficiency gains in inference.
  • Hyperparameter Tuning: While the paper investigates the sensitivity to certain hyperparameters (recurrent iterations, attention heads, model depth), optimal choices may still depend on specific system models and application requirements.
  • Future Work: The authors suggest exploring quantization and lower bitwidth representations to further reduce inference time. They also highlight the importance of analyzing and mitigating the model's computational complexity as an avenue for future research. Additionally, the generalizability of the BRT architecture to other frequency bands and purely far-field models is noted, suggesting potential for broader applications.

Summary updated

Abstract

The integration of terahertz communications and ultra-massive multiple-input multiple-output (UM-MIMO) systems in 6G networks is motivated by their ability to enable unprecedented data rates, mitigate spectrum congestion, and enhance overall network performance. However, the enlarged antenna apertures and higher carrier frequencies in these systems increase the Rayleigh distance, causing users to span both the near-field and conventional far-field regions. Accurate spatial precoding thus requires exact channel estimation at the base station - a task made more challenging by the hybrid coexistence of near- and far-field effects and the limited number of digital chains available in hybrid beamforming architectures. In this paper, we propose a block recurrent transformer model to address this challenge. We demonstrate that a single transformer block equipped with state memory can be trained once and then iteratively applied for hybrid-field channel estimation. Furthermore, we train the model such that it generalizes to wireless channels with varying scatterer distances, different numbers of propagation paths, and wideband operation. Simulation results show that the proposed method achieves performance gains of approximately 5 dB and 7.5 dB in normalized mean squared error (NMSE) over state-of-the-art solutions in narrowband and wideband scenarios, respectively.
BibTeX
@article{artemasov2025recurrent,
  author = {Dmitry Artemasov and Alexander Shmatok and Kirill Andreev and Alexey A. Frolov and Manjesh Kumar Hanawal and Nikola Zlatanov},
  title = {{Recurrent Transformer-Based Near-and Far-Field THz Wideband Channel Estimation for UM-MIMO}},
  journal = {IEEE Access},
  year = {2025},
  doi = {10.1109/access.2025.3639689},
}
Figure