Bayesian Modelling of Community-Based Multidimensional Trust in Participatory Sensing under Data Sparsity

Matteo Venanzi, W. T. Luke Teacy, Alex Rogers, Nick R. JenningsIJCAI 2015

Introduction

Participatory sensing (PS) utilizes the ubiquity of smartphones to collect large-scale environmental data, such as Wi-Fi signal strength, radiation levels, or traffic conditions. Despite its potential, PS systems are often plagued by two significant issues: noise/bias and data sparsity. Human participants use diverse hardware with varying degrees of calibration, and most contributors provide only a small number of measurements. Traditional models often fail to establish a reliable "track record" for users with sparse data. This paper presents CBACE (Community-Based Bayesian Aggregation for Crowdsourced Estimates), a hierarchical Bayesian model designed to address these challenges by leveraging latent community structures.

Methodology: The CBACE Model

The core innovation of CBACE is the assumption that users belong to latent communities with shared reliability profiles. By modeling these communities, the system can "borrow" information from the group to interpret the sparse data of an individual user.

The model distinguishes between two key components of reliability:

  1. Trust ($t_k$): A measure of how accurately a user reports their own sensor's precision (e.g., whether they are overconfident or conservative).
  2. Bias ($b_k$): Systematic errors inherent in the user's hardware or measurement technique (e.g., a GPS offset).
Plate notation of the CBACE hierarchical Bayesian model, illustrating the relationships between community priors, user-specific trust/bias, and observed measurements.

CBACE employs Variational Message Passing (VMP) for efficient inference, allowing it to scale to large datasets. An extension, CBACE+Bias, specifically accounts for systematic offsets in continuous, real-valued data, which is more complex than the discrete labeling tasks handled by prior community-based models.

Experimental Evaluation

The model was evaluated using both real-world and synthetic datasets. The real-world evaluation utilized the OpenSignal Wi-Fi dataset, comprising 7,464 reports from 49 devices across 208 hotspots. Synthetic datasets were used to test the model's resilience under extreme sparsity and its ability to automatically determine the optimal number of communities via marginal log-likelihood analysis.

Results and Discussion

CBACE demonstrates significant improvements over existing state-of-the-art methods like MaxTrust. On the OpenSignal dataset, CBACE achieved a 16.6% improvement in accuracy.

The model's most striking advantage is its performance under data sparsity. By utilizing community-based priors, CBACE can provide accurate estimates even when users contribute very few reports. In simulations of extreme sparsity, CBACE outperformed competitors by up to 49%.

Performance comparison (Mean Error Squared) of CBACE and CBACE+Bias against baselines. Plot (a) shows performance with 100-400 reports, while plot (b) highlights the model's resilience in sparse data scenarios (0-30 reports).

Furthermore, the model successfully identified distinct user groups within the crowd, such as "calibrated" users (51%), "conservative" users (29%), and "uncalibrated" users (20%), providing a nuanced understanding of participant reliability that goes beyond a single trust score.

Conclusion

CBACE offers a robust framework for aggregating crowdsourced data by modeling multidimensional trust and bias through latent communities. Its ability to handle data sparsity makes it particularly suitable for real-world participatory sensing applications where user engagement is often fleeting. Future work will explore dynamic reliability models to account for sensor degradation or user learning over time.

Summary updated

BibTeX
@inproceedings{venanzi2015bayesian,
  author = {Matteo Venanzi and W. T. Luke Teacy and Alex Rogers and Nick R. Jennings},
  title = {{Bayesian Modelling of Community-Based Multidimensional Trust in Participatory Sensing under Data Sparsity}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2015},
}
Figure