Learning Non-myopic Power Allocation in Constrained Scenarios

Arindam Chowdhury, Santiago Paternain, Gunjan Verma, Ananthram Swami, Santiago Segarra

Abstract

We propose a learning-based framework for efficient power allocation in ad hoc interference networks under episodic constraints. The problem of optimal power allocation -- for maximizing a given network utility metric -- under instantaneous constraints has recently gained significant popularity. Several learnable algorithms have been proposed to obtain fast, effective, and near-optimal performance. However, a more realistic scenario arises when the utility metric has to be optimized for an entire episode under time-coupled constraints. In this case, the instantaneous power needs to be regulated so that the given utility can be optimized over an entire sequence of wireless network realizations while satisfying the constraint at all times. Solving each instance independently will be myopic as the long-term constraint cannot modulate such a solution. Instead, we frame this as a constrained and sequential decision-making problem, and employ an actor-critic algorithm to obtain the constraint-aware power allocation at each step. We present experimental analyses to illustrate the effectiveness of our method in terms of superior episodic network-utility performance and its efficiency in terms of time and computational complexity.

Introduction

In wireless ad hoc networks, multiple transceiver pairs share the same frequency spectrum, leading to mutual interference. Managing this interference to maximize data rates typically involves power allocation algorithms. Most existing deep learning-based solutions are "myopic"—they optimize for the immediate channel state without considering long-term resource constraints, such as limited battery life.

In real-world scenarios, devices operate under time-coupled constraints. A decision to transmit at high power now directly reduces the energy available for future transmissions. Myopic strategies often exhaust batteries during periods of poor channel conditions, missing opportunities for high-throughput transmission when conditions improve. This paper introduces Non-myopic Power Allocation (NMPA), a framework designed to manage power budgets strategically over an entire episode.

Methodology

The NMPA framework utilizes a two-level hierarchical architecture to separate instantaneous interference management from long-term resource strategy.

Hierarchical architecture of the NMPA framework, showing the interaction between the upper-level learner and the lower-level solver.
  1. Lower-Level Solver ($g_\phi$): This component uses a pre-trained UWMMSE (Unfolded Weighted Minimum Mean Squared Error) model. It calculates the optimal power allocation for the current interference graph, assuming an infinite power budget.
  2. Upper-Level Learner ($\pi_\theta$): A Graph Convolutional Neural Network (GCNN) serves as the strategic controller. It observes the current battery levels and network topology to produce a scale factor (0 to 1) that modulates the lower-level's output.

The system is trained as a Markov Decision Process (MDP) using the TD3 (Twin Delayed DDPG) algorithm. The reward function incorporates penalties for constraint violations (e.g., attempting to transmit with no battery), encouraging the model to learn a sustainable, non-greedy policy.

Experimental Setup

The framework was evaluated using simulated wireless environments:

  • Network Topology: 10 transceiver pairs randomly distributed in a 60x60 area.
  • Channel Dynamics: Simulations included path loss and Rayleigh fading.
  • Scenarios: The model was tested on both high-interference ("poor") and low-interference ("good") topologies.
  • Constraints: Initial battery budgets were sampled from a uniform distribution to test generalization across different energy levels.

Results and Discussion

The NMPA framework demonstrates a significant shift in transmission strategy compared to myopic approaches.

Performance metrics of NMPA: (a) cumulative sum-rate over time, (b) learned power scaling policy, and (c) generalization across episode lengths.

Strategic Resource Management

As shown in Figure 3(a), NMPA often exhibits a "waiting" behavior during early time steps if channel conditions are poor. While the myopic model (MPA) spends energy immediately, NMPA conserves it, eventually achieving a higher cumulative sum-rate as it utilizes its budget during better channel realizations. Figure 3(b) visualizes this policy: the scale factor is higher when the achievable rate is high and battery is sufficient, and lower (red) when the rate is low or battery is critical.

Performance and Generalization

NMPA achieved a 15–20% improvement in episodic sum-rate over myopic baselines. Furthermore, the model maintained near-zero constraint violations. Figure 3(c) shows that although trained on 100-step episodes, the model generalizes effectively to episode lengths ranging from 30 to 150 steps, indicating it has learned a robust resource management strategy rather than a fixed sequence of actions.

Computational Efficiency

The total inference time for the hierarchical model is approximately 7ms, making it suitable for real-time applications in dynamic wireless environments.

Conclusion

NMPA successfully addresses the challenge of time-coupled constraints in wireless power allocation by combining the strengths of GCNNs and constrained reinforcement learning. Future work will focus on transitioning from centralized to distributed training, incorporating node mobility, and managing multiple simultaneous constraints such as latency and thermal limits.

Summary updated

Abstract

We propose a learning-based framework for efficient power allocation in ad hoc interference networks under episodic constraints. The problem of optimal power allocation -- for maximizing a given network utility metric -- under instantaneous constraints has recently gained significant popularity. Several learnable algorithms have been proposed to obtain fast, effective, and near-optimal performance. However, a more realistic scenario arises when the utility metric has to be optimized for an entire episode under time-coupled constraints. In this case, the instantaneous power needs to be regulated so that the given utility can be optimized over an entire sequence of wireless network realizations while satisfying the constraint at all times. Solving each instance independently will be myopic as the long-term constraint cannot modulate such a solution. Instead, we frame this as a constrained and sequential decision-making problem, and employ an actor-critic algorithm to obtain the constraint-aware power allocation at each step. We present experimental analyses to illustrate the effectiveness of our method in terms of superior episodic network-utility performance and its efficiency in terms of time and computational complexity.
BibTeX
@inproceedings{chowdhury2023learning,
  author = {Arindam Chowdhury and Santiago Paternain and Gunjan Verma and Ananthram Swami and Santiago Segarra},
  title = {{Learning Non-myopic Power Allocation in Constrained Scenarios}},
  booktitle = {ACSSC},
  year = {2023},
  url = {https://openreview.net/forum?id=9gWx7jtvT2},
}
Figure