LiTformer: Efficient Modeling and Analysis of High-Speed Link Transmitters Using Non-Autoregressive Transformer

Songyu Sun, Xiao Dong, Yanliang Sha, Quan Chen, Cheng ZhuoICCAD 2024 · DOI

TL;DR

This paper introduces LiTformer, a novel Transformer-based model that efficiently and accurately models high-speed link transmitters (TXs), even in complex multi-link scenarios with crosstalk. By leveraging a non-autoregressive Transformer architecture and a unique encoding scheme for link parameters, LiTformer achieves significant speedups over traditional simulation methods while maintaining high accuracy.

What Problem Does This Paper Address?

Modern computing systems, from AI accelerators to 5G infrastructure, rely heavily on high-speed serial links for efficient data transfer. The transmitters (TXs) within these links are critical for signal quality, but their behavior is complex due to nonlinearities, dynamic interactions with transmission lines and receivers, and increasingly, crosstalk from numerous adjacent links in dense chiplet designs. Accurately modeling these TXs is crucial for predicting signal integrity issues and ensuring reliable data transmission. Traditional methods like SPICE simulations are highly accurate but prohibitively slow, while simpler models often lack the necessary accuracy or the ability to incorporate complex link parameters and crosstalk effects. Existing deep learning approaches have struggled with capturing long-range dependencies in nonlinear signals or efficiently integrating non-sequential link parameters.

Methods

The core innovation is LiTformer, a Transformer-based model employing a non-sequential encoder-decoder architecture.

  • Non-Sequential Encoder: This encoder processes various inputs independently:
  • Scalar Features: Link parameters like impedance ($Z_0$), capacitance ($C_L$), voltage ($V_p$), and transmission line characteristics ($S_{tl}$) are standardized and fed through individual Multi-Layer Perceptrons (MLPs).
  • Input Signal Encoding: Instead of processing the raw signal sequence, LiTformer detects transition edges in the input signal. This edge information is then embedded and processed, allowing the model to capture signal dynamics without strict sequentiality.
  • S-parameter Encoding: Frequency-dependent S-parameters of transmission lines are processed through convolutional layers to extract relevant features.
  • Transformer Decoder: A standard Transformer decoder architecture is used, but with a crucial modification:
  • Non-Autoregressive (NAR) Mechanism: During training and inference, the decoder predicts the entire output sequence in parallel, rather than one token at a time. This is achieved through a "randomly masked training" strategy where some output tokens are masked, and the model learns to predict them based on the context.
  • One-Pass Filtered Decoding: For inference, the model generates a full output sequence in a single pass. A Savitzky-Golay filter is then applied to smooth the reconstructed waveform, ensuring efficiency and accuracy.
  • Classification Formulation: The continuous voltage output is mapped to discrete classes, turning the regression problem into a classification task for improved model performance.

Datasets

The authors generated datasets from HSPICE simulations of four commercial high-speed TX designs: two NRZ single-ended (TS1, TS2), one NRZ differential (TD3), and one PAM4 single-ended (TP4). These datasets were created by simulating 2-link and 16-link TX outputs with uniformly sampled link parameters and random 4-symbol (NRZ) or 8-symbol (PAM4) input sequences. A total of 15,000 samples were generated, split into training (12k), validation (1k), and testing (2k) sets.

Key Insights and Ideas

  • Edge-based Signal Encoding: Representing the input signal by its transition edges, rather than the raw sequence, is a clever way to decouple signal dynamics from strict temporal order, enabling efficient integration with non-sequential link parameters.
  • Non-Autoregressive Transformer for Circuit Modeling: Applying NAR Transformers to circuit modeling is a significant step. It overcomes the sequential bottleneck of RNNs/LSTMs, enabling parallel processing for much faster inference, which is critical for complex simulations.
  • Unified Modeling of Intrinsic and Crosstalk: LiTformer can model both the TX's intrinsic output and the crosstalk from other links within a single framework, treating them as different "tasks" (K=0 for intrinsic, K=1 for crosstalk). This holistic approach is essential for accurate multi-link analysis.
  • Classification for Voltage Prediction: Converting the voltage prediction into a classification problem simplifies the learning task and allows for more robust reconstruction of the signal waveform.

Key Results

  • Speedup: LiTformer achieves substantial speedups compared to SPICE:
  • 2-link systems: 148-456x speedup.
  • 16-link systems: 404-944x speedup.
  • Accuracy: Mean relative errors (RE) are very low:
  • Intrinsic outputs: 0.61-1.26%.
  • Crosstalk outputs: 0.48-3.32% (with small absolute errors, indicating minor amplitudes).
  • Versatility: The model supports various TX types (single-ended, differential, PAM4) and data rates (Gbps) with 4-bit signals.
  • Comparison to prior work: LiTformer significantly outperforms FNN and LSTM models in accuracy and speed, especially when link parameters are considered.

Limitations and Future Work

  • Limitations:
    • The paper focuses on 4-bit signals; extending to higher-order modulation schemes (e.g., PAM8) might require adjustments.
  • While LiTformer handles link parameters, the complexity of S-parameter encoding might be a bottleneck for extremely high-frequency or complex channel models.
  • Future Work:
    • The authors suggest LiTformer can be extended to model receivers (RXs) and entire link systems.
  • Further exploration of NAR Transformer architectures for other circuit modeling tasks is implied.
  • Investigating the model's performance with even higher data rates and more complex signal modulation schemes would be valuable.

Summary updated

BibTeX
@inproceedings{sun2024litformer,
  author = {Songyu Sun and Xiao Dong and Yanliang Sha and Quan Chen and Cheng Zhuo},
  title = {{LiTformer: Efficient Modeling and Analysis of High-Speed Link Transmitters Using Non-Autoregressive Transformer}},
  booktitle = {ICCAD},
  year = {2024},
  doi = {10.1145/3676536.3676778},
}
Figure