Efficient Information Updates in Compute-First Networking via Reinforcement Learning with Joint AoI and VoI

Jianpeng Qi, Chao Liu, Chengxiang Xu, Rui Wang, Junyu Dong, Yanwei YuarXiv 2025 · DOI · arXiv

Abstract

This paper introduces AVA (Age-and-Value-Aware), a novel metric designed to balance information freshness with decision-making utility in Compute-First Networking (CFN). By leveraging Reinforcement Learning (PPO) to trigger status updates only when they are semantically significant for task offloading, the proposed framework achieves a 90% to 98% reduction in communication overhead while maintaining high decision accuracy.

Introduction

In Compute-First Networking (CFN), Access Points (APs) must make real-time decisions on where to offload client tasks based on the resource availability (e.g., CPU threads) of edge servers. This environment presents a fundamental communication trade-off. Relying on stale information leads to poor offloading decisions, such as sending tasks to overloaded servers. Conversely, frequent status updates to ensure freshness can congest the network, wasting valuable bandwidth and energy.

Traditional metrics like Age of Information (AoI) focus solely on temporal freshness. However, freshness does not always equate to utility; a stale report is still accurate if the server state hasn't changed. There is a critical need for metrics that jointly consider temporal freshness and semantic consistency.

System Architecture and Methodology

The authors propose a closed-loop system where a decision node manages information updates between service nodes and APs. The system is modeled as a Markov Decision Process (MDP) and optimized using Proximal Policy Optimization (PPO).

System architecture showing the interaction between service nodes, APs, and the decision node within the CFN environment.

The AVA Metric

The core contribution is the AVA (Age-and-Value-Aware) metric, which is the product of two key components:

  1. Normalized QAoI (Query Age of Information): Measures freshness specifically at the moments a client requests a task, ignoring idle periods.
  2. Consistency Metric ($R$): Quantifies the mathematical discrepancy between the server's actual capacity and the AP's observed capacity.

By combining these, the system prioritizes updates when information is both old and likely to be incorrect, ensuring that communication resources are spent only when the "Value of Information" (VoI) is high.

Reinforcement Learning for Information Updates

The RL agent observes the system state—including current server capacity, the AP's last known capacity, and the time elapsed since the last update—to decide whether to trigger a refresh. This behavior is visualized in the temporal analysis below, which shows how the agent learns to ignore fluctuations that don't impact the "Query" context while reacting to significant state changes.

Temporal analysis of system status, AoI, and update decisions, illustrating how updates are triggered in response to client requests and state discrepancies.

Experimental Results

The framework was evaluated using an ns-3 simulation environment interfaced with OpenAI Gymnasium. Testing across various traffic patterns (Deterministic and Uniformly Distributed) yielded several key insights:

  • Massive Overhead Reduction: The AVA-based policy reduced update frequency by over 90% on average compared to AoI-minimal baselines. In stochastic traffic scenarios, the reduction reached up to 98.34%.
  • Maintained Decision Accuracy: Despite the drastic reduction in messaging, the ratio of correct offloading decisions remained equal to or better than traditional "chatty" methods.
  • Context-Awareness: The agent successfully learned to increase update frequency when servers approached capacity (where errors are costly) and remained quiet during stable or idle periods.

Conclusion

The AVA metric and RL-based update policy demonstrate that information staleness is not inherently detrimental if the underlying data remains semantically valid for the task at hand. By focusing on the intersection of query timing and state consistency, CFN systems can drastically reduce network overhead. Future work will explore extending this model to multi-source, multi-destination environments and incorporating complex queuing dynamics.

Summary updated

BibTeX
@misc{qi2025efficient,
  author = {Jianpeng Qi and Chao Liu and Chengxiang Xu and Rui Wang and Junyu Dong and Yanwei Yu},
  title = {{Efficient Information Updates in Compute-First Networking via Reinforcement Learning with Joint AoI and VoI}},
  year = {2025},
  eprint = {2505.06025},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2505.06025},
}
Figure