Mathematical Foundations of Semantic Communication

35 concepts28 papers1 proposal

In 1948, Claude Shannon established the bedrock of modern digital communication by intentionally ignoring meaning. Shannon’s classical information theory treats communication as an engineering problem of reproducing a sequence of symbols perfectly, regardless of what those symbols represent. For decades, this "bit-pipe" paradigm sufficed. However, as we enter the era of the Internet-of-Everything and distributed edge intelligence, networks face unprecedented data volumes that threaten to overwhelm physical bandwidth limits, as noted by Wei Xu et al. in Edge Learning for B5G Networks with Distributed Signal Processing: Semantic Communication, Edge Computing, and Wireless Sensing.

To overcome these bottlenecks, researchers are developing Semantic Communication (SemCom). Instead of transmitting raw bits perfectly, SemCom aims to transmit only the underlying meaning or task-relevant intent of the data. Yet, shifting from bit-perfect transmission to meaning-perfect transmission requires rewriting the rules of information theory. The central challenge of this research direction is building a rigorous mathematical foundation equivalent to Shannon's framework—one that can formally define semantic entropy, semantic channel capacity, and rate-distortion bounds.

The Geometry of Meaning: Synonymous Mappings

To understand how semantic communication compresses data so drastically, consider how it redefines an information source. In classical theory, every distinct bit sequence is treated as a unique message. In semantic theory, the source is modeled as a many-to-one mapping, where multiple distinct data sequences correspond to the exact same underlying meaning.

Classical View (Shannon):
[Sequence A] ──> Unique Message A
[Sequence B] ──> Unique Message B
[Sequence C] ──> Unique Message C

Semantic View (Many-to-One):
[Sequence A] ┐
[Sequence B] ┼─> Shared Semantic Meaning (Equivalence Class)
[Sequence C] ┘

By grouping data patterns that carry identical intent into "synonymous sets," a system can avoid transmitting redundant variations. As long as the receiver reconstructs a message that falls within the correct synonymous set, the transmission is considered semantically lossless.

The authors of Semantic Arithmetic Coding using Synonymous Mappings leverage this insight to design practical coding algorithms that approach the theoretical limits of semantic entropy. Similarly, Shuai Ma 0002 et al. formalize this many-to-one structure in A Theory for Semantic Channel Coding With Many-to-One Source, providing a mathematical framework to handle the inherent ambiguity and context-dependency of meaning across different knowledge bases. To ground this mathematically, Haizi Yu et al. in Semantic Compression with Information Lattice Learning revisit Shannon's own obscure 1953 work on information lattices, using lattice theory to formally represent semantic equivalence and optimize abstract data compression.

Defining the Speed Limits: Semantic Rate-Distortion

In classical lossy compression, the Rate-Distortion function dictates the absolute minimum bit rate required to reconstruct a signal within a given error threshold (such as mean squared error). For semantic communication, we need a way to measure distortion not in terms of altered pixels or flipped bits, but in terms of altered meaning.

To address this, Chuan Huang 0001 et al. introduce the Semantic Rate-Distortion Function (SRDF) in Fundamental Limitation of Semantic Communications: Neural Estimation for Rate-Distortion. The SRDF establishes the theoretical "speed limits" for meaning-based transmission by calculating the minimum bit rate required to preserve both raw observations and their associated semantic states. Because the true statistical distribution of semantic meaning is often mathematically intractable, the authors develop a neural-network-based estimator (NESRD) to compute these bounds directly from data.

Formulating these limits requires moving past naive extensions of classical entropy. As Deniz Gunduz et al. explain in Beyond Transmitting Bits: Context, Semantics, and Task-Oriented Communications, early attempts at semantic information theory suffered from a paradox where logically contradictory statements were assigned the highest informational value. Overcoming these theoretical discrepancies is vital for creating robust operational definitions of semantic entropy that possess practical engineering utility.

Goal-Oriented Networks and Shared Knowledge

Semantic communication does not operate in isolation; meaning is inherently tied to a specific goal or context. In Goal-Oriented Communications for the IoT: System Design and Adaptive Resource Optimization, Mattia Merluzzi et al. demonstrate that when a network optimizes for a downstream task (such as edge AI inference or remote control) rather than data reconstruction, resource efficiency can improve by up to two times.

To achieve this efficiency without transmitting raw data, the transmitter and receiver must share a common context. The authors of A Joint Communication and Computation Design for Semantic Wireless Communication with Probability Graph propose using a Probability Graph as a shared knowledge base.

[Transmitter] ──(Sparse Semantic Update)──> [Receiver]
      │                                           │
[Probability Graph]                       [Probability Graph]
(Omits Redundant Data)                    (Infers Missing Details)

By utilizing the graph to omit redundant information during transmission, the receiver can perform local computation to "infer" missing details, trading off heavy network bandwidth for local edge computation.

The Standardization Challenge

While individual architectures show massive promise, the broader adoption of semantic communication faces a significant hurdle: the lack of standardization. In classical networks, the bit serves as a universal currency, and the bit-error rate provides an objective performance metric.

As emphasized by Xidong Mu et al. in Exploiting Semantic Communication for Non-Orthogonal Multiple Access and Loc X. Nguyen et al. in A Contemporary Survey on Semantic Communications:Theory of Mind, Generative AI, and Deep Joint Source-Channel Coding, the absence of a standardized, mathematically rigorous metric for quantifying the transfer of meaning makes it incredibly difficult to objectively compare different protocols.

Furthermore, Yili Jin et al. point out in Generative AI for Multimedia Communication: Recent Advances, An Information-Theoretic Framework, and Future Opportunities that meaning is often tied to specific AI model architectures, preventing interoperability. Because semantic information is inherently subjective and context-dependent, Angeliki Alexiou et al. argue in Beyond Shannon Communications-A Paradigm Shift to Catalyze 6G that establishing universally accepted mathematical definitions of semantic entropy and effectiveness remains the most critical open challenge for the field, essential for ensuring future interoperability across hardware manufacturers and network operators.

Key Foundational Work

An Algorithm of Limited Syntax Based on Language Universals (1973) — contributed 1 concept elements to this direction.
Computational Linguistics and Linguistic Theory (1973) — contributed 1 concept elements to this direction.
On a Certain Distribution of Semantic Units (1965) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_semantic_information_theory_3beb278c,
  title = {semantic information theory},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/semantic-information-theory/rdr_3beb278c71a446018d6b64202880901d},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-21},
}

This exact version (version 1, frozen text):

@misc{lacuna_semantic_information_theory_3beb278c_v1,
  title = {semantic information theory},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/semantic-information-theory/rdr_3beb278c71a446018d6b64202880901d/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-21},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake