Abstract
In this chapter, we consider the semantic joint source channel coding. Similar to the classic information theory, we can tie together two basic methods of semantic communication: semantic source coding and semantic channel coding and prove the semantic source channel coding theorem. Compared with the classic communication system, the code rate range of semantic communication can be further extended. This point reveals the significant potential of semantic coding.
Introduction
Classical information theory, pioneered by Claude Shannon, focuses on the technical problem of transmitting symbols accurately over a noisy channel. The fundamental limit for reliable transmission is defined by the condition that the source entropy $H(U)$ must be less than the channel capacity $C$. However, as we move toward 6G and AI-driven applications, the objective is increasingly to convey meaning (semantics) rather than exact bit sequences.
This paper addresses the theoretical gap by providing a mathematical framework for Semantic Joint Source Channel Coding (SJSCC). By formalizing the relationship between semantic source and channel coding, the authors establish the fundamental bounds for how much semantic information can be reliably transmitted, even in regimes where classical methods fail.
The SJSCC Framework
The authors extend classical source-channel coding theorems into the semantic domain using several key mathematical constructs. Central to this is the concept of a Synonymous Mapping ($f$), which maps raw source data $U$ to a semantic variable $\tilde{U}$, effectively grouping different symbols that carry the same meaning.
The system model, as illustrated above, incorporates the following metrics:
- Semantic Entropy ($H_s$) and Rate-Distortion ($R_s$): New metrics that measure information content and compression limits based on semantic significance rather than raw data statistics.
- Semantic Capacity ($C_s$): A measure of the maximum rate at which semantic information can be reliably transmitted over a channel.
- Separation Principle: The authors prove that semantic source coding and semantic channel coding can be designed separately and then integrated without losing asymptotic optimality.
Breaking the Shannon Limit
The most striking insight of this work is that semantic communication can operate in regimes where $H(U) > C$. Because multiple data sequences can represent the same underlying meaning, the semantic entropy $H_s(\tilde{U})$ is generally lower than the classical entropy $H(U)$.
This "semantic gap" represents a significant opportunity: in "impossible" intervals where a classical system would fail to reconstruct the source, a semantic system can still successfully convey the intended message. The paper demonstrates that the fundamental structure of information theory remains robust; the semantic versions of the theorems mirror classical ones but offer a more powerful toolkit for modern communication.
Fundamental Theorems
The paper culminates in two primary theorems that define the "gold standard" for semantic communication:
- Theorem 10.1 (Lossy Case): Establishes that if the semantic rate-distortion $R_s(D)$ is less than the semantic capacity $C_s$, there exists a code that keeps semantic distortion below a target threshold $D$.
- Theorem 10.2 (Lossless Case): Proves that if the semantic entropy $H_s(\tilde{U})$ is less than $C_s$, the probability of a semantic error tends to zero as the code length $n$ increases.
Mathematically, the conditions for reliable semantic communication are summarized as:
- Lossless: $H_s(\tilde{U}) \leq R \leq C_s$
- Lossy: $R_s(D) \leq R \leq C_s$
Discussion and Future Work
While this work provides a rigorous mathematical proof of existence for these codes, it does not prescribe specific algorithms for constructing the synonymous mapping ($f$) or the semantic distortion measure ($d_s$) for complex, real-world data like high-definition video.
The authors suggest that these theoretical results provide the necessary foundation for future practical implementations. Future research will likely leverage machine learning techniques, such as Deep Joint Source Channel Coding, to approximate these theoretical semantic functions in practical hardware, paving the way for 6G networks that prioritize meaning over bits.
Summary updated