Abstract
The rapid advancement of generative artificial intelligence has spurred innovative approaches to semantic communication, giving rise to a new paradigm known as generative semantic communication (GSC). The integration of flexible cross-modal semantic extraction with generative capability-driven semantic inference substantially enhances semantic compression efficiency, demonstrating significant promise under communication resource constraints. Nonetheless, the stringent dependence on high computational power and the resulting latency continue to present major challenges, thereby limiting the feasibility of large-scale deployment. To address these challenges, we propose a novel GSC framework named FAST, which stands for flexible and adaptive semantic transmission. To accommodate limited computational resources, we propose a sequential semantic extraction method, where a temporal prompt engineering module orchestrates the distillation and transmission of key semantic units. Correspondingly, we introduce a sequential conditional denoising module at the receiver, which adapts the diffusion-based reconstruction to the progressively received input. To enhance overall task performance in multi-user semantic transmission, we propose a semantic-aware resource allocation method that optimizes bandwidth dynamically based on a joint consideration of semantic dependencies, user-level task priorities, and instantaneous channel conditions. Extensive experiments demonstrate that the proposed architecture achieves system precision comparable to conventional GSC systems while significantly reducing transmission latency and improving overall efficiency. These results confirm its enhanced potential for deployment in multi-user GSC scenarios with stringent communication and computational constraints.
Introduction
As we move toward the 6G era and massive IoT integration, traditional data transmission methods are increasingly hitting a "spectrum wall." Generative Semantic Communication (GSC) offers a promising solution: instead of transmitting raw pixels or high-bitrate data, the transmitter sends "semantic prompts" (such as text descriptions or edge maps) that a Generative AI (GAI) model at the receiver uses to reconstruct the original content.
While GSC is highly efficient at compression, it faces a significant bottleneck: latency. Conventional GSC systems typically follow an "all-or-nothing" approach, waiting for the entire semantic prompt to arrive before beginning the computationally intensive reconstruction process. In resource-constrained, multi-user environments—such as digital twin systems or autonomous robot fleets—this leads to substantial delays and inefficient use of edge computing resources. The FAST framework addresses this by transforming GSC into a "stream-and-refine" pipeline.
The FAST Framework
The FAST (Flexible and Adaptive Semantic Transmission) framework decomposes the transmission and reconstruction process into three coordinated modules designed to optimize both computational and communication resources.
1. Temporal Prompt Engineering
Instead of extracting and sending all semantic units simultaneously, an RL agent (trained via Proximal Policy Optimization) determines the optimal sequence for extraction. By employing Invalid Action Masking, the agent efficiently learns which parts of a prompt are most critical for the final reconstruction, prioritizing high-impact semantic units to minimize initial latency.
2. Sequential Conditional Denoising
At the receiver, the system does not wait for the full prompt. It initiates the diffusion denoising process as soon as the first semantic units arrive. When subsequent information arrives mid-process, a Semantic Difference Calculation Subunit identifies how the noise space must be adjusted to incorporate new data without restarting the generation from scratch.
3. Semantic-aware Resource Allocation
In multi-user scenarios, bandwidth is not allocated solely based on Signal-to-Noise Ratio (SNR). The system dynamically shifts resources based on user task priority and the "semantic dependency" of the units currently being transmitted, ensuring that the most critical information across the network is prioritized.
Key Insights: Semantic Prioritization
A core finding of the FAST framework is the hierarchical importance of semantic types. Statistical analysis of the RL agent's behavior reveals a consistent prioritization of nouns (the foundation of the scene), followed by verbs (actions), and finally adjectives or styles (fine-grained details).
As illustrated in the figure above, the order of extraction is vital. If an action (like "Jumping") is extracted before the subject ("Dog"), the generative model may fail to produce a coherent image. The FAST framework's ability to "steer" image generation mid-stream allows it to refine the output as specific semantic units arrive, effectively overlapping computation with transmission.
Experimental Evaluation
The framework was evaluated using the DIFFUSIONDB dataset, utilizing approximately 0.2 million prompts segmented into semantic units (NOUN, ADJECTIVE, STYLE, VERB, etc.) via the spaCy NLP library. Experiments were conducted using Stable Diffusion v1-4 on NVIDIA RTX 4090 GPUs.
Key performance metrics include:
- Latency Reduction: FAST achieved a 21% reduction in residual system latency compared to baseline GSC methods.
- Precision Gains: System precision (measured by CLIP score) improved by 29% over baselines lacking sequential denoising and semantic-aware allocation.
- Efficiency: The overall system efficiency was approximately 1.6 times higher than conventional GSC architectures.
Conclusion
The FAST framework demonstrates that generative AI communication can be made practical for real-world, resource-constrained environments. By breaking the "wait-and-generate" cycle, FAST enables low-latency, high-precision semantic transmission. Future work will focus on expanding the framework to support diverse modalities such as video and 3D point clouds, as well as enhancing security against prompt-based vulnerabilities.
Summary updated