Semantic communication systems send the meaning of a message rather than its exact bits. A text transmitter might send a handful of knowledge-graph facts (entity–relation–entity triples) instead of a full sentence, and let the receiver rewrite the sentence. Because nobody can check "meaning" directly, these systems are trained and scored with automatic proxies: cosine similarity between sentence embeddings, embedding similarity between images, or composite scores such as the Metric of Semantic Similarity (MSS). Reinforcement learning (RL) is now used to decide what to transmit under a bandwidth budget, with one of these proxies as the reward. The language-generation literature shows what happens when a policy is pushed hard against a single learned or embedding-based score: it finds the score's blind spots. The semantic-communication literature has never checked whether its transmitters do the same.
This proposal makes that check the centerpiece. We train RL transmitters against individual semantic metrics and measure how far each metric's gains fail to carry over to held-out checks of meaning that the policy never sees. We then test whether a small set of complementary rewards, one policy conditioned on the channel quality and the bandwidth budget, removes that gap across the whole rate–fidelity range. The expected result is either a demonstration that current semantic metrics are gameable as training signals, together with a reward design that is not, or a clean negative result showing they are safe to optimize. Either would change how semantic transmitters are trained and reported.
Core Claim
A transmitter trained by RL to maximize a single embedding-based semantic score will learn to send content that keeps the score high while losing the information a receiver actually needs. The main candidates are generic, high-frequency triples that the reconstructor can paraphrase fluently, while rare entities and polarity-bearing relations (negations, "not", "reduces" versus "increases") get dropped, because embedding cosine is plausibly insensitive to both. We call the difference between the gain on the trained metric and the gain on held-out meaning checks the hacking gap. The claim has two parts. The first is that this gap is large and grows with RL pressure under the reward used by Performance Optimization for Semantic Communications: An Attention-Based Reinforcement Learning Approach, which trains an attention-based proximal policy optimization (PPO) agent on a scalar MSS. The second is that a composite reward, built from signals whose blind spots do not overlap and conditioned on channel state through control tokens, closes most of the gap at equal bandwidth.
What is new is the audit and the evidence about the reward design, not the use of RL in semantic communication. The closest work, the APPO system above, optimizes a scalar MSS and reports a two-fold MSS improvement and a 41.3% token reduction on AGENDA. It never asks whether MSS gains reflect meaning that survives. Deep Learning Enabled Semantic Communication Systems introduced BERT sentence similarity as an evaluation metric, not as an RL reward under optimization pressure. Distortion Resilience for Goal-Oriented Semantic Communication treats semantic distortion as a fixed threshold for image classification, so it never exposes a text metric to a policy that can exploit it.
Research Program
The transmitter. The testbed follows the APPO structure so results transfer directly to it. At each step the transmitter sees the triples extracted from a source text, the current channel signal-to-noise ratio (SNR), and a symbol budget. It chooses which triples to send and how much redundancy to give each one, and each triple costs channel symbols in proportion to its protection. Triples cross an additive-white-Gaussian-noise or Rayleigh-fading channel. They pass through a digital path whose error model follows the integer-quantized latents of Semantic Communication on Digital Wireless Communication Systems, so corrupted triples come out as plausible wrong values rather than random noise. A fixed pre-trained graph-to-text model rebuilds the sentence at the receiver. Freezing the receiver matters: any gaming must come from what the transmitter chooses to send, which makes the analysis clean.
The rewards. We train separate single-reward policies on BERT sentence similarity (as in DeepSC), on MSS (as in APPO), and on a bidirectional entailment score between source and reconstruction. Each is paired with the same symbol-cost penalty. These are the objects of the audit.
The composite, conditioned policy. Borrowing the method of Tailoring Self-Rationalizers with Multi-Reward Distillation, which adapts quantile-based reward conditioning so that one small model learns output distributions indexed by control tokens for several reward quantiles, we bin each reward component into quantiles. The bins are embedding similarity, bidirectional entailment, rare-entity recall (weighted by inverse corpus frequency) and symbol cost. The transmitter is trained with the corresponding tokens prepended, alongside tokens for the SNR bucket. At deployment the network requests an operating point, such as low cost with high entailment at low SNR, by choosing tokens. No downstream critic runs in the loop, which also sidesteps the latency concern DRGO raises about waiting on downstream models.
The composite is chosen for complementary failure modes rather than for coverage. Embedding similarity rewards topical closeness. Entailment penalizes flipped polarity and invented facts. Rare-entity recall penalizes the retreat to generic content. Cost penalizes padding.
The audit. Meaning is scored by checks that no policy is ever trained on. For each source text we build cloze questions from its gold triples, one question per triple (for example, "Method X was evaluated on ___"). A fixed extractive question-answering model, distinct from every reward model, answers them from the reconstructed sentence. Answer accuracy serves as our proxy for what Shannon and Weaver called the effectiveness level of communication, meaning whether the receiver can act on the message. We also count polarity errors between source and reconstruction, and collect human pairwise judgments on a stratified sample.
The hacking gap for a policy $\pi$ is
$$G(\pi) = \frac{M_{\text{train}}(\pi) - M_{\text{train}}(\pi_0)}{\sigma_{\text{train}}} - \frac{M_{\text{held}}(\pi) - M_{\text{held}}(\pi_0)}{\sigma_{\text{held}}},$$
where $M_{\text{train}}$ is the metric the policy was trained on, $M_{\text{held}}$ is held-out question-answering accuracy, $\pi_0$ is the supervised starting transmitter, and each $\sigma$ is that metric's standard deviation across source texts under $\pi_0$. This puts gains on a common scale. We track $G$ across training checkpoints and across increasing KL-penalty relaxation. If gaming exists, $G$ should rise as optimization pressure increases.
Baselines
The naive reference point is separate source and channel coding: Huffman or Brotli compression followed by Reed–Solomon or Turbo coding with adaptive modulation, run over the same channels. The whole sentence is sent whenever the symbol budget allows and it is truncated otherwise. DeepSC reports that this pipeline collapses sharply at low SNR while its learned system's sentence similarity "often converg[es] to 1". That comparison was made on Europarl, not AGENDA, so we re-run it on our setup rather than borrow the figure. This baseline tells us whether any learned transmitter is worth having at a given budget.
The strongest existing method is the APPO transmitter trained on scalar MSS, reimplemented on AGENDA and DocRED with the same frozen receiver, channel model, symbol budgets and number of environment interactions as our policy. The reported figures from that paper are a "two-fold improvement in the total MSS compared to standard communication networks" and a "41.3% reduction in the number of tokens". Both are relative, not absolute, and there is no reported result on Lacuna for held-out question-answering accuracy or any meaning check beyond MSS. Beating it means matching its MSS within noise while achieving higher held-out question-answering accuracy and fewer polarity errors at every tested SNR and budget. A composite policy that loses on MSS but wins on the held-out checks also counts, provided human judges prefer it.
The dynamic-scheduling competitor is DORB: Dynamically Optimizing Multiple Rewards with Bandits, which uses an Exp3 bandit to decide which metric to optimize at each stage. It reports 18.68 BLEU-4 against 17.88 for cross-entropy on SQuAD question generation, and 33.58 against 33.97 TER on WebNLG. We run DORB's single-level bandit over our same four reward components with the same interaction budget. This tests whether scheduling alone captures the benefit of conditioning.
Two ablations isolate the key ingredients. The first removes the SNR tokens while keeping the reward tokens, which tests whether channel conditioning matters. The second keeps the conditioning but drops rare-entity recall and entailment from the composite, which tests whether complementary blind spots, rather than having more rewards, drive the result.
Why not just…?
Why not just use a weighted sum of the same rewards? This is the most serious threat, and we do not dismiss it. The direction summary and DORB argue that one component with larger variance can dominate a static sum, but DORB's reported gains over static baselines are small (under one BLEU point). We therefore give the weighted sum a fair chance. We grid-search the weights separately for each operating point with the same total compute as the conditioned policy, then compare against the frontier traced by those tuned policies. If the tuned sums match the conditioned policy on held-out accuracy, the contribution narrows to two things: the audit, and the finding that the choice of complementary components is what matters. We would report it that way. The one remaining advantage of conditioning would be operational, since one model covers every operating point.
Why not just replace the embedding metric with a better single one, such as entailment or question answering? It may simply move the blind spot. An entailment-only reward can be met by sending very little: a short, safe reconstruction entails nothing false. This is the same brevity loophole that forced the length penalties in Reinforced Extractive Summarization with Question-Focused Rewards. The entailment-only arm of the audit tests this directly. If its hacking gap is near zero and its held-out accuracy matches the composite, the simpler design wins and we will say so.
Why not just keep supervised training and skip RL? Supervised transmitters cannot game a metric, so they are the natural zero-gap control, and they are our $\pi_0$. What supervision cannot do is trade content against a variable symbol budget and channel state, because there are no labels for which triples to drop. Reward Engineering for Generating Semi-structured Explanation likewise found that supervised fine-tuning alone produced structurally weak outputs that RL with composite rewards fixed. If $\pi_0$ plus a simple rule, such as sending the triples with the rarest entities first, matches the RL policies on held-out accuracy at equal budget, then RL is not needed here. That rule is included as a baseline.
Proposed Experiments
The primary experiment uses AGENDA, whose 40,000 abstracts come paired with knowledge graphs, and DocRED for longer multi-sentence relations. Policies are trained over SNRs from 0 to 18 dB on both channel types, at budgets of 25%, 50% and 75% of the full triple set. The first decisive check is the audit itself: the hacking gap $G$ for the BERT-similarity and MSS policies at convergence, with bootstrap confidence intervals over source texts, and its trend across checkpoints. A gap indistinguishable from zero would mean existing semantic metrics are safe to optimize at this scale. That is a publishable negative result, and it would redirect the rest of the program toward the weighted-sum question alone.
The second decisive check compares the composite conditioned policy against tuned APPO, tuned weighted sums and DORB on held-out question-answering accuracy, polarity-error rate and human preference, all at matched symbol cost. We plot full rate–accuracy frontiers per SNR rather than single points, so that apparent wins cannot come from choosing a favorable operating point.
The third check is transfer. We move the frozen trained transmitter to WebNLG triples, with a WebNLG-trained receiver, without retraining, and test whether the ordering of hacking gaps holds. A metric's blind spots should belong to the metric, not to the dataset.
As a stretch probe, we apply the same audit to caption-based image transmission on MSCOCO, following the pipeline of Vision Language Models for Massive MIMO Semantic Communication. That system reports PSNR of 8.28–8.78 dB alongside high image-embedding similarity. There, a caption-selection policy trained on image-embedding similarity is checked against held-out object-presence detection on the regenerated image, which tests whether the same gaming appears when the receiver is a generative model.
References
- Performance Optimization for Semantic Communications: An Attention-Based Reinforcement Learning Approach
- Deep Learning Enabled Semantic Communication Systems
- Distortion Resilience for Goal-Oriented Semantic Communication
- Semantic Communication on Digital Wireless Communication Systems
- Tailoring Self-Rationalizers with Multi-Reward Distillation
- DORB: Dynamically Optimizing Multiple Rewards with Bandits
- Reinforced Extractive Summarization with Question-Focused Rewards
- Reward Engineering for Generating Semi-structured Explanation
- Vision Language Models for Massive MIMO Semantic Communication