Cell Selection with Deep Reinforcement Learning in Sparse Mobile Crowdsensing

Leye Wang, Wenbin Liu, Daqing Zhang, Yasha Wang, En Wang, Yongjian Yang

Abstract

Sparse Mobile Crowdsensing (MCS) aims to reduce data collection costs by sensing only a subset of cells and inferring the rest. This paper presents DR-Cell, a Deep Reinforcement Learning (DRL) framework for optimal cell selection. By employing a Deep Recurrent Q-Network (DRQN) with LSTM layers, DR-Cell captures spatial and temporal correlations to minimize sensing costs while meeting data quality constraints. Experimental results on real-world datasets show a 15% reduction in required samples compared to state-of-the-art heuristic methods.

Introduction

Mobile Crowdsensing (MCS) leverages smartphone sensors for urban monitoring while minimizing participant recruitment costs. The core challenge lies in selecting an optimal subset of "cells" (geographic sub-areas) to sense such that the remaining data can be accurately inferred. Traditional heuristic-based methods, such as Query-By-Committee (QBC), often fail to directly optimize for the minimum number of sensed cells under strict quality guarantees.

The sensing process involves iteratively choosing cells to collect data, which then informs the inference of unsensed regions.

Sequential cell selection and data inference process in Sparse MCS.

As shown in the figure above, the system transitions through cycles, progressively filling the data matrix with collected and inferred values. This paper introduces DR-Cell, a DRL framework that treats cell selection as a sequential decision-making problem to maximize sensing efficiency.

The DR-Cell Framework

DR-Cell utilizes a Deep Recurrent Q-Network (DRQN) to learn an optimal sensing policy. The architecture maps the current state of the sensing grid to the most beneficial next action.

The DR-Cell reinforcement learning architecture mapping states to optimal cell selection actions.

The framework consists of several key components:

  1. State Representation: The agent observes a "Cell Selection Matrix" representing the sensing history.
  2. Action Space: The agent selects the next cell to be sensed in the current cycle.
  3. Reward Mechanism: A reward function balances the cost of sensing against a significant bonus for achieving the target inference quality threshold.
  4. Recurrent Architecture: By incorporating Long Short-Term Memory (LSTM) layers, the DRQN captures both spatial dependencies between neighboring cells and temporal patterns over successive cycles.

To address data scarcity, the authors employ transfer learning. A model pre-trained on a data-rich "source" task (e.g., temperature sensing) is fine-tuned for a "target" task (e.g., humidity), leveraging shared spatial characteristics of environmental variables.

Experimental Evaluation

The authors evaluated DR-Cell on two real-world datasets: Sensor-Scope (environmental data from the EPFL campus) and U-Air (air quality data from Beijing). These datasets provide the ground truth necessary to simulate sparse sensing scenarios.

Results demonstrate that DR-Cell reduces the required number of sensed cells by up to 15% compared to QBC while maintaining identical inference quality. In temperature monitoring scenarios, DR-Cell maintained 90% quality using only ~22% of available cells. Furthermore, transfer learning significantly improved performance in scenarios with limited training data, highlighting the framework's generalizability.

Conclusion

DR-Cell represents a significant advancement in Sparse MCS by shifting from heuristic-based selection to an optimized, learning-based policy. By integrating recurrent neural networks with reinforcement learning, the system effectively exploits the spatio-temporal correlations inherent in urban data. Future work will focus on addressing the "cold start" problem through online learning and incorporating non-uniform sensing costs.

Summary updated

Abstract

Sparse Mobile CrowdSensing (MCS) is a novel MCS paradigm where data inference is incorporated into the MCS process for reducing sensing costs while its quality is guaranteed. Since the sensed data from different cells (sub-areas) of the target sensing area will probably lead to diverse levels of inference data quality, cell selection (i.e., choose which cells of the target area to collect sensed data from participants) is a critical issue that will impact the total amount of data that requires to be collected (i.e., data collection costs) for ensuring a certain level of quality. To address this issue, this paper proposes a Deep Reinforcement learning based Cell selection mechanism for Sparse MCS, called DR-Cell. We properly model the key concepts in reinforcement learning including state, action, and reward, and then propose to use a deep recurrent Q-network for learning the Q-function that can help decide which cell is a better choice under a certain state during cell selection. Experiments on various real-life sensing datasets verify the effectiveness of DR-Cell over the state-of-the-art cell selection mechanisms in Sparse MCS by reducing up to 15% of sensed cells with the same data inference quality guarantee.
BibTeX
@inproceedings{wang2018cell,
  author = {Leye Wang and Wenbin Liu and Daqing Zhang and Yasha Wang and En Wang and Yongjian Yang},
  title = {{Cell Selection with Deep Reinforcement Learning in Sparse Mobile Crowdsensing}},
  booktitle = {ICDCS},
  year = {2018},
  url = {https://openreview.net/forum?id=ZkYI2GGkdP},
}
Figure