Abstract
Optimal transport has numerous applications, particularly in machine learning tasks involving generative models. In practice, the transportation process often encounters an information bottleneck, typically arising from the conversion of a communication channel into a rate-limited bit pipeline using error correction codes. While this conversion enables a channel-oblivious approach to optimal transport, it fails to fully exploit the available degrees of freedom. Motivated by the emerging paradigm of generative communication, this paper examines the problem of channel-aware optimal transport, where a block of i.i.d. random variables is transmitted through a memoryless channel to generate another block of i.i.d. random variables with a prescribed marginal distribution such that the end-to-end distortion is minimized. With unlimited common randomness available to the encoder and decoder, the source-channel separation architecture is shown to be asymptotically optimal as the blocklength approaches infinity. On the other hand, in the absence of common randomness, the source-channel separation architecture is generally suboptimal. For this scenario, a hybrid coding scheme is proposed, which partially retains the generative capabilities of the given channel while enabling reliable transmission of digital information. It is demonstrated that the proposed hybrid coding scheme can outperform both separation-based and uncoded schemes.
Introduction
Traditional communication systems are built on the premise of reliable bit reconstruction. By utilizing error-correction codes, engineers create a "bit pipeline" that effectively hides channel noise from the source coder. However, the emergence of generative AI—such as GANs and diffusion models—has introduced a new objective: generative communication. In this paradigm, the receiver's goal is not necessarily to reconstruct the exact source bits, but to generate a sample that follows a specific target distribution while minimizing distortion. This shift introduces a "perception constraint" that challenges long-standing information-theoretic assumptions.
The Problem: Channel-Aware Optimal Transport
The core challenge in generative communication is the requirement for randomness. Generative models need a source of stochasticity to produce diverse and accurate samples. In a communication system, this randomness can be sourced from two places: a shared random seed (common randomness) or the communication channel itself.
This paper introduces the framework of Channel-Aware Optimal Transport. The authors investigate how to transmit a block of i.i.d. random variables through a memoryless channel to generate a reconstruction with a prescribed marginal distribution, minimizing end-to-end distortion. This framework moves beyond treating the channel as a simple bit-pipe and instead considers it an active component of the generative process.
Architectural Approaches
The study evaluates three primary architectures for this task:
- Separation-based Coding: Following Shannon’s classical approach, the channel is treated as a bit-pipe. The source is compressed into bits, and the decoder uses local randomness to generate the target distribution.
- Uncoded Transmission: The source is mapped directly to the channel input. This "analog" approach allows channel noise to pass through to the receiver, potentially serving as the required randomness.
- Hybrid Coding: A novel scheme proposed by the authors that combines digital and analog components. It transmits a digital refinement while simultaneously "harnessing" the channel's inherent stochasticity to satisfy the generative model's need for randomness.
The Role of Common Randomness
A pivotal finding of the research is the impact of common randomness ($Q$) shared between the encoder and decoder.
- With Common Randomness: When a shared seed is available, the source-channel separation architecture is asymptotically optimal. The decoder can use the shared seed to generate the required distribution without incurring additional distortion penalties.
- Without Common Randomness: In the absence of a shared seed, separation becomes strictly suboptimal. The system suffers from a "double distortion" penalty: once during lossy compression and again when the decoder must add independent noise to meet the distribution requirement.
Channel Noise as a Resource
One of the most provocative insights of this work is that channel noise should be viewed as a feature rather than a bug. In scenarios without common randomness, the channel's own stochasticity can serve as the entropy source for the generative decoder. By using uncoded or hybrid schemes, the system "delegates" part of the generative process to the channel itself.
This theoretical result provides a rigorous explanation for the success of Deep Joint Source-Channel Coding (Deep JSCC). While traditional codecs struggle when the "bit-pipe" breaks down, Deep JSCC models often degrade gracefully because they naturally exploit the channel's characteristics rather than trying to suppress them entirely.
Key Results and Comparisons
The authors provide mathematical proofs for both Binary Symmetric Channels (BSC) and Additive White Gaussian Noise (AWGN) channels using Bernoulli and Multivariate Gaussian distributions:
- Theorem 1 (Optimality of Separation): Confirms that with unlimited common randomness, separation holds.
- Theorem 2 (Hybrid Superiority): Demonstrates that a hybrid scheme can strictly outperform both pure separation and pure uncoded schemes by balancing digital reliability with analog noise utilization.
- Gaussian Case: In vector Gaussian settings, the hybrid scheme achieves a lower distortion-power tradeoff than any linear or separation-based scheme. At low power regimes, the hybrid scheme's advantage is particularly pronounced, with distortion gains over separation proportional to the square root of the power.
Conclusion and Future Work
This work establishes that the "death of separation" is a fundamental reality for generative communication when common randomness is unavailable. By framing the problem as channel-aware optimal transport, the authors provide a roadmap for designing next-generation communication systems that treat the channel as an integral part of the generative model.
Future research directions include expanding the theory to handle bandwidth mismatch (where the number of channel uses differs from the number of source symbols) and investigating "one-shot" versions of the problem for real-world, low-latency generative systems.
Summary updated