Towards a generic framework for trustworthy spatial crowdsourcing

Cyrus ShahabiMobiDE 2013 · DOI

TL;DR

This paper introduces a framework for "spatial crowdsourcing" (SC), a novel approach to data collection using mobile phone users to perform location-based tasks. The authors highlight two critical challenges: scalability in matching numerous tasks to workers efficiently and trust in validating the collected data, proposing to address these through reputation systems and optimized task assignment.

What Problem Does This Paper Address?

The paper tackles the growing need for efficient and scalable data collection in urban, social, and spatiotemporal contexts, especially in situations where traditional methods fail (e.g., during disasters or censorship). The authors observe the ubiquitous nature of smartphones, their increasing capabilities (sensors, bandwidth), and the sheer number of users, presenting a massive opportunity for crowdsourced data gathering. However, they identify two major roadblocks to realizing this potential:

  1. Scalability: How to efficiently match a large and dynamic pool of requesters' tasks with a similarly fluid set of mobile workers in real-time. This involves handling the constant influx of new tasks, workers, and the completion or expiration of existing ones.
  2. Trust: How to ensure the credibility and accuracy of the data contributed by individual workers. Without a robust trust mechanism, the collected data may be unreliable, rendering the entire SC system ineffective.

Methods

The authors propose a framework for spatial crowdsourcing, exemplified by their proof-of-concept system, GeoCrowd. The core of their approach lies in addressing the scale and trust issues:

  • Scalability: They identify task assignment as the primary bottleneck. While task assignment problems are well-studied, the spatial nature of SC (requiring physical travel to specific locations) makes it unique. They plan to study a cloud-based distributed approach for better scalability.
  • Trust: They extend existing trustworthy computing concepts by focusing on the data contributor. Their approach involves maintaining a reputation score per worker and a confidence level for every spatial task. The idea is to assign tasks to enough workers such that their aggregated reputation meets the task's confidence threshold. A key challenge here is that optimal task assignment for trust heavily depends on the geographical locations of workers and tasks, making it a complex optimization problem, which they believe to be NP-hard. They aim to prove this and develop heuristics that leverage spatial properties.

Datasets

Not discussed in the paper.

Key Insights and Ideas

  • Smartphones as powerful, ubiquitous sensors: The paper emphasizes the transformative potential of leveraging the massive user base and advanced capabilities of smartphones for data collection, moving beyond traditional methods.
  • Spatial Crowdsourcing as a new paradigm: SC is presented as a distinct approach that requires workers to physically visit locations, differentiating it from general crowdsourcing.
  • Task Assignment is the critical bottleneck for scalability: The dynamic nature of SC tasks and workers makes efficient, real-time assignment a significant challenge.
  • Trust in the human contributor is paramount: While data integrity during transmission is addressed by existing tech, the paper highlights the novel challenge of trusting the individual worker's contribution.
  • Reputation and confidence scores are key to trust: The proposed system aims to build trust by quantifying worker reliability and task certainty through scores.
  • Geographical location is central to both scale and trust optimization: The spatial aspect of SC is not just a feature but a core constraint that complicates optimization problems for both task assignment and trust aggregation.

Key Results

This is a position paper outlining a research direction and a proposed framework. Therefore, specific quantitative results are not presented. The paper focuses on defining the problem and outlining the proposed approach.

Limitations and Future Work

  • Complexity of Trust Optimization: The authors acknowledge that optimizing task assignment for trust aggregation is a complex problem, likely NP-hard. They plan to prove this and develop heuristic solutions.
  • Scalability of Task Assignment: While they propose a cloud-based distributed approach, the specifics and effectiveness of this solution are part of their future work.
  • Real-world validation: The paper describes a proof-of-concept system (GeoCrowd) but does not present results from extensive real-world deployment or evaluation.

Future work will focus on proving the NP-hardness of the trust optimization problem, developing efficient heuristics for it, and exploring cloud-based distributed architectures for scalable task assignment.

Summary updated

BibTeX
@inproceedings{shahabi2013towards,
  author = {Cyrus Shahabi},
  title = {{Towards a generic framework for trustworthy spatial crowdsourcing}},
  booktitle = {MobiDE},
  year = {2013},
  doi = {10.1145/2486084.2486085},
}
Figure