Goal-Oriented Scheduling in Sensor Networks with Application Timing Awareness

Josefine Holm, Federico Chiariotti, Anders E. Kalør, Beatriz Soret, Torben Bach Pedersen, Petar Popovski

TL;DR

This paper introduces a novel approach to sensor network scheduling that prioritizes specific client application goals over simply minimizing overall system error. By using a query-aware Deep Reinforcement Learning (DRL) framework, the system dynamically learns to poll sensors that provide the most relevant information for diverse and potentially conflicting client requests, leading to significant improvements in application-specific performance.

What Problem Does This Paper Address?

Traditional sensor networks often focus on collecting data to estimate the overall state of a system, aiming to minimize metrics like Mean Squared Error (MSE). However, in modern applications like Industry 4.0, multiple clients might interact with the same sensor network, each with unique and evolving needs. These clients may require different functions of the system state (e.g., the maximum value among sensors, or the count of sensors within a specific range) at different times. Simply minimizing the overall system MSE doesn't guarantee optimal performance for these diverse, time-sensitive client requests. This paper addresses the challenge of designing a "pragmatic" communication system that can intelligently schedule sensor data acquisition to satisfy these varied and dynamic application-level goals, going beyond traditional "semantic" communication.

Methods

The core of the proposed solution lies in a query-aware Deep Reinforcement Learning (DRL) framework. The system models the problem as a Partially Observable Markov Decision Process (POMDP), where the edge node (acting as the scheduler) needs to decide which sensor to poll at each time step. The state of the POMDP includes the estimated system state and its uncertainty (from a Kalman filter), as well as the "age" of information for each client's query.

Key methodological contributions include:

  • Dynamic Value of Information (VoI): Instead of a static VoI based on overall MSE, the paper proposes a dynamic VoI that considers the specific function of the state requested by each client and the timing of their queries.
  • Deep Q-Network (DQN): A DQN architecture is employed to approximate the optimal policy for the scheduling problem. This is necessary because the state space is too large for traditional tabular methods.
  • Kalman Filter: A standard Kalman filter is used for state estimation and tracking, providing the necessary information about the system's state and its uncertainty.
  • Query Process Modeling: Client query arrivals are modeled using Markov chains, allowing for both periodic and memoryless (geometrically distributed) query patterns.

Datasets

The paper does not introduce new datasets. Instead, it relies on simulated scenarios to evaluate the proposed framework. These simulations involve generating sensor data based on a linear dynamic system and creating various query arrival patterns (periodic, memoryless, and mixed) for multiple clients with different query functions (state, sample mean, sample variance, maximum, count range).

Key Insights and Ideas

  • Pragmatism Over Raw Data: The most crucial insight is that optimizing for specific application goals (pragmatism) can be far more effective than simply minimizing overall data error. Different clients require different "views" of the data, and the scheduler must adapt to these needs.
  • Query Timing is Crucial: The timing of client queries significantly impacts the optimal scheduling strategy. A pragmatic scheduler can tolerate temporary increases in MSE for inactive queries if it means minimizing error for an upcoming, critical query.
  • DRL for Dynamic Scheduling: DRL proves to be a powerful tool for learning complex, dynamic scheduling policies in scenarios with multiple, diverse, and time-varying client demands.
  • Non-Linear Functions Matter: The paper highlights that simple metrics like average Aol are insufficient when clients request non-linear functions of the state (e.g., maximum value). The optimal policy for such queries can be drastically different.
  • Trade-offs are Inevitable: When multiple clients with conflicting goals are present, the system must make trade-offs. The RL approach learns to balance these competing demands to achieve the best overall outcome according to the defined reward function.

Key Results

  • The proposed query-aware DRL solution can outperform naive approaches (like static VoI or Maximum Age First) by 15-20% in terms of application-specific performance.
  • In periodic query scenarios, the RL policy achieved a lower overall cost (weighted MSE) compared to other policies, demonstrating its ability to balance different query types.
  • The paper shows that the optimal polling strategy is highly dependent on the specific query functions. For instance, the strategy for a "maximum" query differs significantly from that for a "count range" query.
  • While the MAF policy provides a baseline by minimizing average Age of Information (Aol), it often results in higher MSE for specific queries compared to the RL-based approach.
  • The complexity of the query arrival process (deterministic vs. stochastic) significantly impacts the learning process and the effectiveness of the RL strategy. Unpredictable query arrivals make it harder for the RL agent to learn beyond simple greedy strategies.

Limitations and Future Work

  • Single Sensor Polling: The current framework assumes the edge node polls only one sensor at a time. Future work could explore scenarios where multiple sensors can be polled simultaneously.
  • Pull-Based Communication: The system relies on a pull-based communication model. Extending this to scenarios where sensors can transmit urgent information proactively could be beneficial.
  • Richer Information Types: The current work focuses on numeric sensor values. Future research could investigate scheduling for more complex data types like images or point clouds, which are relevant for applications like cooperative driving or robot swarm management.
  • Advanced Estimators: The paper uses a Kalman filter for state estimation. Exploring the use of more advanced, data-driven estimators (e.g., deep networks) that do not require prior knowledge of system dynamics is suggested.
  • Integration with Control Systems: Combining remote estimation with control systems to create fully task-oriented communication systems is a promising avenue for future research.
  • Training Complexity: The paper notes that mixed deterministic and stochastic query processes are the most difficult to train for, suggesting that more sophisticated training strategies or longer training times might be needed for such complex scenarios.

Summary updated

Abstract

Taking inspiration from linguistics, the communications theoretical community has recently shown a significant recent interest in pragmatic , or goal-oriented, communication. In this paper, we tackle the problem of pragmatic communication with multiple clients with different, and potentially conflicting, objectives. We capture the goal-oriented aspect through the metric of Value of Information (VoI), which considers the estimation of the remote process as well as the timing constraints. However, the most common definition of VoI is simply the Mean Square Error (MSE) of the whole system state, regardless of the relevance for a specific client. Our work aims to overcome this limitation by including different summary statistics, i.e., value functions of the state, for separate clients, and a diversified query process on the client side, expressed through the fact that different applications may request different functions of the process state at different times. A query-aware Deep Reinforcement Learning (DRL) solution based on statically defined VoI can outperform naive approaches by 15-20%.
BibTeX
@article{holm2025goal,
  author = {Josefine Holm and Federico Chiariotti and Anders E. Kalør and Beatriz Soret and Torben Bach Pedersen and Petar Popovski},
  title = {{Goal-Oriented Scheduling in Sensor Networks with Application Timing Awareness}},
  journal = {IEEE Transactions on Communications},
  year = {2025},
  url = {https://openreview.net/forum?id=6Keu4HS8lc},
}
Figure