Suppose a transmitter does not know the current quality of its wireless channel. One way to cope is to split each transmission into layers: a heavily protected base layer that almost always gets through, and lighter refinement layers that get through only when the channel happens to be good. This "broadcast approach" trades peak rate for graceful degradation. When the base layer carries time-critical status updates and the refinement layers carry bulk data, the transmitter faces a real two-sided conflict. More power on the base layer keeps the receiver's information fresh. More power on the refinements raises throughput. The best power split depends on the operator's preference, on how stale the receiver's information currently is, and on the fading statistics. In practice those statistics are rarely known exactly.
The proposal is that the layered structure itself is the learning signal. Each layer is decoded only if the channel gain exceeds a threshold that the transmitter sets through its power split. A per-layer acknowledgment therefore tells the transmitter which interval the gain fell into. One block with three layers is three threshold queries on the unknown fading distribution, where a single-layer link gives only one. We plan to build an online learner that uses this censored feedback, together with the receiver's current age of information, to recover the entire throughput-versus-freshness trade-off curve. The curve is indexed by the freshness target, so the learner produces a whole family of good operating points rather than one. We will measure how much this gains over the best age-blind policy known to date and over generic deep multi-objective reinforcement learning given the same number of channel uses.
Core Claim
Three claims, in order of how much they depend on new work.
The first is structural and almost free. Long-run average throughput and long-run average age are both linear in the occupation measure, which is the long-run frequency of each (age, action) pair. Truncate the age at a large value, discretize the layer power splits, and allow randomization across blocks. The set of achievable (throughput, age) pairs is then a linear image of a polytope, and so it is convex. This means the exact front for a known fading distribution is a parametric linear program in the age target $\alpha$. It also means the zero-duality-gap result of Constrained Reinforcement Learning Has Zero Duality Gap holds here trivially, without function approximation. That paper shows constrained RL is convex in occupation measures; we reuse its formulation as the oracle.
This first claim directly contradicts the research file's suggestion that SIC interference makes these fronts non-convex. Successive interference cancellation (SIC) is the receiver's procedure of decoding a layer, subtracting it, and decoding the next. The per-block rate region under SIC may well be non-convex. Randomizing over blocks removes that. Non-convexity survives only if the operator forbids randomization, and we will check this explicitly rather than assume it.
The second claim is the real contribution. With the fading distribution unknown, a model-based learner reaches the oracle front with regret that grows more slowly when it receives per-layer acknowledgments than when it receives base-layer acknowledgments alone. The learner estimates the probability that the gain lands in each interval between layer thresholds, adds optimism bonuses, and re-solves the parametric linear program. Choosing where to put the thresholds is then partly an exploration decision. We tentatively expect regret on the order of $\sqrt{T}$ in the number of blocks $T$, following standard concentration for interval probabilities. We expect the constant to shrink with the number of layers, because each block refines several intervals at once.
The third claim is the size of the prize. Age-aware layered policies beat the age-blind optimum by a margin that grows as the freshness target tightens. The gap grows further under time-correlated fading, where the previous block's decoded-layer count is a delayed, partial reading of the current channel.
Relation to Prior Work
The closest work is Throughput Maximization with an Average Age of Information Constraint in Fading Channels, which motivates this proposal and supplies its strongest baseline. Under no transmitter channel knowledge, it puts status updates in the base layer and bulk data in refinement layers. It then restricts to age-independent stationary randomized policies (AI-SRP), which yields a convex program. It proves at least 50% of optimal throughput and reports 90% to 95% in simulation. It assumes the fading distribution is known, it discards the age state, and it solves one constraint level at a time. We keep its channel model and its layer-to-traffic mapping. We change three things: we keep the age state, we learn the distribution from acknowledgments, and we return the whole front.
The Broadcast Approach in Communication Networks supplies the method and a constraint. It is a survey that optimizes continuous-layer power with Euler–Lagrange equations for expected rate. It also reports that two or three discrete layers capture most of the gain on single-antenna channels, and that observation justifies our small discrete action space.
Broadcast Approach to Uplink NOMA: Queuing Delay Analysis analyzes queuing delay for layered multi-user uplink and shows that layering avoids the delay blow-up of single-layer outage schemes. An Information-Theoretic View of Mixed-Delay Traffic in 5G and 6G shows that superposition beats orthogonal puncturing for mixed latency-critical and bulk traffic. Both motivate the setting and fix split parameters offline. Neither learns, and neither tracks per-block state.
On the multi-objective side, three methods learn a whole front with one model. PD-MORL conditions a single network on a preference vector. Pareto Policy Adaptation uses implicitly differentiated common-ascent directions. Multi-Objective Reinforcement Learning with Continuous Pareto Frontier Approximation uses a parametric policy manifold. All three are model-free and generic, and none exploits the threshold-query structure of layered feedback. We use PD-MORL as the deep baseline.
Optimal Scalarizations for Sublinear Hypervolume Regret shows that linear scalarization misses concave regions of a front, and that random hypervolume scalarizations achieve optimal hypervolume regret. Because our front is convex under randomization, its warning mostly does not bite here. We keep hypervolume scalarization only as an ablation over how the learner allocates exploration across $\alpha$.
Research Program
The model is block fading. A transmitter sends $L \in \{1,2,3\}$ superposed layers. Layer 1 carries a status update and layers 2 and above carry bulk data. The receiver knows the channel and acknowledges the index of the highest layer it decoded. The state is the receiver's age of information, truncated at $A_{\max}$. In the correlated-fading variant, the state also includes the last decoded-layer count. An action is a discretized power split across layers, which fixes the decoding thresholds. Let $\mu(s,a)$ be the occupation measure over states $s$ and actions $a$, and let $p$ be the vector of interval probabilities of the channel gain. The oracle front is
$$ R^*(\alpha) = \max_{\mu \ge 0} \sum_{s,a} \mu(s,a)\, r_p(a) \quad \text{s.t.} \quad \sum_{s,a} \mu(s,a)\, s \le \alpha, \ \ \mu \text{ stationary under } P_p, $$
where $r_p(a)$ is the expected bulk throughput of action $a$ under $p$, and $P_p$ is the age transition kernel. Both $r_p$ and $P_p$ are linear in $p$.
The learner keeps counts of which interval each block's gain fell into, which the acknowledgments reveal up to the chosen thresholds. It forms optimistic estimates of $p$. At each epoch it solves the linear program above for a grid of $\alpha$ values, so the operator can switch freshness targets at any time without retraining. The analysis will bound regret and constraint violation relative to $R^*(\alpha)$ uniformly over the grid, and will make the dependence on $L$ explicit.
The stretch goal is the two-user uplink NOMA setting from Broadcast Approach to Uplink NOMA: Queuing Delay Analysis, with a third objective, transmit energy. Here the state space grows and the deep baseline may become competitive.
Baselines
The naive reference is single-layer transmission at a fixed outage target with retransmission. Its rate and outage target are tuned by grid search for each $\alpha$, using the true distribution. Broadcast Approach to Uplink NOMA: Queuing Delay Analysis reports that layered schemes give "substantially lower average sum delays," but no absolute number is reported in the file. We will compute this baseline ourselves.
The strongest existing method is AI-SRP from Throughput Maximization with an Average Age of Information Constraint in Fading Channels. It reports 90% to 95% of optimal throughput on Rayleigh fading, with a proven 50% floor. We give it the true distribution, which is an advantage our learner lacks, and solve it at every $\alpha$ on the grid. Beating it means two things. First, the age-aware oracle must exceed AI-SRP by a margin larger than simulation noise at tight $\alpha$. Second, our learner must cross AI-SRP's front within a stated number of blocks despite starting ignorant.
The deep competitor is PD-MORL (PD-MORL), conditioned on $\alpha$ through a Lagrangian reward. It gets the same state, the same acknowledgment feedback, the same block budget, and an equal hyperparameter-search budget. It reports 25% larger hypervolume than prior MORL methods on MuJoCo. However, no reported result was found on Lacuna for any learned policy on the layered throughput–freshness problem.
Three ablations isolate the key ingredients. The first gives our learner base-layer-only acknowledgments, which isolates the value of layers as probes. The second gives it age-blind actions, which isolates the value of state. The third replaces uniform $\alpha$ coverage with random hypervolume scalarizations, which tests whether exploration allocation matters on a convex front. The upper bound throughout is the known-distribution linear program oracle.
Why not just…?
Why not just use a weighted sum of throughput and age? Because the front is convex under randomization, a weighted sum, or equivalently the Lagrangian, does recover every point on it. We use exactly that inside the linear program, so this alternative is adopted rather than argued against. It would fail only if the operator forbids randomization across blocks. We will compute the deterministic-only front to show whether concave dents ever appear in that case.
Why not just solve the linear program once with a measured fading distribution? If the distribution is known and stationary, that is our oracle and it wins by definition. The question is how many blocks of operation it takes to know the distribution well enough. The learning curves against the oracle, and the per-layer versus base-only ablation, answer this directly. If a short pilot phase with single-layer probing matches our learner, the probe idea adds little and we will report that.
Why not just deploy AI-SRP? It is simple and already within 5% to 10% of optimal in the reported setting. If the age-aware oracle beats it by less than about 2% across the whole $\alpha$ range under independent fading, state-awareness is not worth its complexity there. The proposal would then rest on the correlated-fading regime and on the learning claim.
Why not just train a deep multi-objective agent? It needs no modeling. However, it ignores the fact that an acknowledgment is a threshold query on a scalar distribution, so we expect it to need many more blocks. The equal-budget comparison settles this. In the energy-augmented multi-user setting it may close the gap, and we will report that honestly.
Proposed Experiments
All experiments use simulated Rayleigh block fading as in Throughput Maximization with an Average Age of Information Constraint in Fading Channels. We add a Gauss–Markov correlated variant whose correlation runs from 0 to 0.95, and use $L$ from 1 to 3. The metrics are throughput at each $\alpha$, constraint violation, hypervolume of the (throughput, negative age) front against a fixed reference point, and cumulative regret relative to the oracle as a function of blocks.
There are three decisive checks. The first is whether the oracle's margin over AI-SRP is material at tight $\alpha$, and whether it grows with channel correlation. The second is whether regret with per-layer acknowledgments falls measurably below regret with base-only acknowledgments, with the gap widening as $L$ increases. If it does not, the probe idea fails. The third is whether randomized fronts are convex while deterministic-only fronts show dents, which would settle where nonlinear scalarization is actually needed.
References
- Constrained Reinforcement Learning Has Zero Duality Gap
- Throughput Maximization With an Average Age of Information Constraint in Fading Channels
- The Broadcast Approach in Communication Networks
- Broadcast Approach to Uplink NOMA: Queuing Delay Analysis
- An Information-Theoretic View of Mixed-Delay Traffic in 5G and 6G
- PD-MORL: Preference-Driven Multi-Objective Reinforcement Learning Algorithm
- Pareto Policy Adaptation
- Multi-Objective Reinforcement Learning with Continuous Pareto Frontier Approximation
- Optimal Scalarizations for Sublinear Hypervolume Regret