Token Communication in the Era of Large Models: An Information Bottleneck-Based Approach

Hao Wei, Wanli Ni, Wen Wang, Wenjun Xu, Dusit Niyato, Ping Zhang

Abstract

This letter proposes UniToCom, a unified token communication paradigm that treats tokens as the fundamental units for both processing and wireless transmission. Specifically, to enable efficient token representations, we propose a generative information bottleneck (GenIB) principle, which facilitates the learning of tokens that preserve essential information while supporting reliable generation across multiple modalities. By doing this, GenIB-based tokenization is conducive to improving the communication efficiency and reducing computational complexity. Additionally, we develop $\sigma$-GenIB to address the challenges of variance collapse in autoregressive modeling, maintaining representational diversity and stability. Moreover, we employ a causal Transformer-based multimodal large language model (MLLM) at the receiver to unify the processing of both discrete and continuous tokens under the next-token prediction paradigm. Simulation results validate the effectiveness and superiority of the proposed UniToCom compared to baselines under dynamic channel conditions. By integrating token processing with MLLMs, UniToCom enables scalable and generalizable communication in favor of multimodal understanding and generation, providing a potential solution for next-generation intelligent communications.

Introduction

Traditional wireless communication systems have long focused on the accurate transmission of bits. However, as we enter the era of Large Language Models (LLMs), a new paradigm called Semantic Communication (SemCom) has emerged. SemCom shifts the focus from raw data reconstruction to the transmission of high-level "meanings." Despite its promise, existing SemCom systems often rely on continuous, high-dimensional features that lack the structural granularity required to interface seamlessly with modern token-based AI architectures like GPT-4 or LLaVA.

The UniToCom (Unified Token Communication) framework addresses these challenges by redefining the fundamental unit of transmission as the "token." By aligning the communication protocol with the internal representations of Multimodal Large Language Models (MLLMs), UniToCom aims to solve the efficiency bottleneck of transmitting rich multimodal data over constrained wireless channels.

The UniToCom Framework

UniToCom introduces a token-centric pipeline that integrates source and channel coding into a unified process. The system maps input data—whether text, images, or audio—directly into latent tokens optimized for both transmission and AI reasoning.

The UniToCom architecture featuring tokenization, adaptive channel coding, and a receiver-side MLLM.

Key components of the framework include:

  • Generative Information Bottleneck (GenIB): This principle guides the compression process by finding a "minimal sufficient" representation. It balances the compression rate against the receiver's ability to generate or reason about the original content.
  • $\sigma$-GenIB: To prevent "variance collapse"—a common issue in generative AI where latent spaces become too narrow—$\sigma$-GenIB imposes fixed variance constraints. This ensures representational diversity and stability during autoregressive modeling.
  • Receiver-side MLLM: The receiver employs a 1.4B parameter causal Transformer. It treats the communication task as a "next-token prediction" problem, using specialized heads (e.g., diffusion heads for continuous data) to reconstruct high-fidelity outputs.

Experimental Results and Analysis

The framework was evaluated across three tasks: Visual Question Answering (VQA) on CLEVR, text-to-image generation on MS-COCO, and Automatic Speech Recognition (ASR) on LibriSpeech.

Performance comparison of UniToCom against traditional and vanilla SemCom schemes across various SNR levels.

As shown in the simulation results, UniToCom demonstrates significant robustness to noise. In low Signal-to-Noise Ratio (SNR) regimes, where traditional methods (such as JPEG/AMR-WB with LDPC codes) often fail, UniToCom maintains high performance across all modalities. Specifically, the GenIB approach achieved superior semantic fidelity, as evidenced by lower FID scores in image generation and higher accuracy in VQA tasks compared to "Vanilla" SemCom and VAE-based tokenization.

The "Goldilocks" Zone of Compression

An important insight from the study is the existence of an optimal token sequence length. While one might assume more tokens lead to better results, excessively long sequences can introduce redundant noise that impairs model accuracy. For instance, a token length of 32 was found to be a "sweet spot" for image tasks, balancing information density with computational efficiency.

Conclusion

UniToCom represents a significant step toward integrating communication and intelligence. By treating tokens as a universal currency, the system achieves high efficiency and scalability. While the computational demands of a 1.4B parameter MLLM at the receiver are currently high, the framework provides a robust foundation for next-generation intelligent communications. Future work will focus on optimizing these models for edge deployment and expanding to more complex modalities like video.

Summary updated

Abstract

This letter proposes UniToCom, a unified token communication paradigm that treats tokens as the fundamental units for both processing and wireless transmission. Specifically, to enable efficient token representations, we propose a generative information bottleneck (GenIB) principle, which facilitates the learning of tokens that preserve essential information while supporting reliable generation across multiple modalities. By doing this, GenIB-based tokenization is conducive to improving the communication efficiency and reducing computational complexity. Additionally, we develop $σ$-GenIB to address the challenges of variance collapse in autoregressive modeling, maintaining representational diversity and stability. Moreover, we employ a causal Transformer-based multimodal large language model (MLLM) at the receiver to unify the processing of both discrete and continuous tokens under the next-token prediction paradigm. Simulation results validate the effectiveness and superiority of the proposed UniToCom compared to baselines under dynamic channel conditions. By integrating token processing with MLLMs, UniToCom enables scalable and generalizable communication in favor of multimodal understanding and generation, providing a potential solution for next-generation intelligent communications.
BibTeX
@article{wei2026token,
  author = {Hao Wei and Wanli Ni and Wen Wang and Wenjun Xu and Dusit Niyato and Ping Zhang},
  title = {{Token Communication in the Era of Large Models: An Information Bottleneck-Based Approach}},
  journal = {IEEE Wireless Communications Letters},
  year = {2026},
  doi = {10.1109/lwc.2025.3622628},
}
Figure