Abstract
In Internet of Things (IoT) systems, maintaining data freshness is critical for real-time monitoring and control. This paper addresses the challenge of minimizing the Age of Information (AoI) while considering the energy constraints of battery-powered devices and the nonlinear dynamics of physical processes. We propose a distributed reinforcement learning (RL) framework based on the QMIX algorithm. By integrating physics-aware sampling policies with a centralized-training-decentralized-execution architecture, the system optimizes both sampling at the device level and resource allocation at the base station. Experimental results using real-world pollution data demonstrate that our approach reduces AoI by up to 33.9% and energy consumption by 35.1% compared to traditional uniform sampling methods.
Introduction
The proliferation of IoT devices has enabled real-time monitoring of diverse physical processes, from environmental sensing to industrial automation. In these applications, the value of data is often tied to its "freshness," quantified by the Age of Information (AoI). However, achieving low AoI is complicated by two primary factors: the limited energy reserves of IoT devices and the scarcity of wireless communication resources. Most existing research assumes simplistic periodic sampling, which fails to capture the volatile, nonlinear nature of real-world physical dynamics. This paper introduces a framework that adapts sampling and transmission strategies to the underlying physics of the monitored environment.
Methodology
The proposed solution treats the problem as a joint optimization of the Sampling Policy at the IoT device and the Device Selection at the Base Station (BS).
Physics-Aware AoI Modeling
Unlike standard AoI metrics that rely solely on time elapsed, this model incorporates the dynamics of the physical process. Using Nyquist-Shannon sampling theory, the authors derive a relationship between the variation frequency of the physical signal and the required sampling rate. If a device samples at a rate lower than the signal's Nyquist frequency, the AoI is penalized more heavily to reflect the increased estimation error and loss of tracking accuracy.
Distributed QMIX Framework
To manage the coordination between multiple devices and the BS, the researchers employ the QMIX multi-agent RL algorithm. The architecture follows a centralized-training-decentralized-execution (CTDE) paradigm:
- Local Agents: Each IoT device operates a local Deep Q-Network (DQN) to make sampling decisions based on local observations of its energy state and the physical process.
- Mixing Network: During training, a mixing network at the BS aggregates local Q-values into a global joint action-value function. This ensures that devices learn cooperative behaviors, such as prioritizing transmissions for devices with the most critical data updates.
Experimental Results
The framework was evaluated using a real-world dataset of PM 2.5 air pollution levels from Peking University. This dataset provides a challenging, nonlinear testbed for tracking accuracy, capturing the complex fluctuations of air quality in Beijing.
Performance Comparison
The QMIX-based approach was compared against standard DQN and uniform periodic sampling. As shown in the convergence plots, the QMIX framework achieves higher rewards and faster convergence by effectively managing the trade-off between data freshness and energy expenditure.
Key performance metrics include:
- AoI Reduction: The system achieved a 17.8% reduction in total AoI compared to standard DQN and a 33.9% reduction compared to uniform sampling.
- Energy Efficiency: Total energy consumption was reduced by 13.2% relative to DQN and 35.1% relative to uniform sampling.
- Tracking Accuracy: The QMIX approach demonstrated superior reconstruction of the original PM 2.5 signal, accurately capturing pollution spikes that were missed by other methods.
Conclusion
By explicitly modeling physical dynamics and employing a cooperative multi-agent RL framework, this research provides a robust solution for real-time IoT monitoring. The QMIX-based approach successfully balances the competing demands of data freshness and energy conservation. Future work will explore the scalability of this framework in massive IoT networks and its application to mobile base stations, such as those mounted on unmanned aerial vehicles (UAVs).
Summary updated