Abstract
Modern Artificial Intelligence relies on massive amounts of labelled data, often sourced through crowd labelling. However, performing this over wireless networks introduces significant communication overhead and resource management challenges. This paper introduces a framework for wireless crowd labelling that jointly optimizes data encoding rates, annotator clustering, and spectrum allocation. By balancing the trade-offs between compression distortion and the need for redundant labelling, the proposed Joint Annotator-and-Sub-channel Allocation (JASA) algorithm maximizes labelling throughput—the total number of accurately labelled objects—under strict radio and human resource constraints.
Introduction: The Challenge of Wireless Crowd Labelling
The "fuel" for modern AI is high-quality labelled data, but the labelling process remains a laborious and expensive bottleneck. While outsourcing tasks to a "crowd" of mobile users is a scalable solution, doing so over wireless networks creates a fundamental three-way tradeoff:
- Encoding Rate: Compressing data (e.g., images or video) saves bandwidth but introduces distortion, which increases the likelihood of human labelling errors.
- Repetition Labelling: To mitigate errors caused by distortion or human fallibility, the same object must be sent to multiple annotators. A "majority vote" then determines the final label.
- Resource Allocation: Both wireless spectrum (sub-channels) and the number of available annotators are finite resources that must be managed efficiently.
The central problem is determining how to distribute objects to specific annotators, at what compression level, and on which frequency to maximize throughput without sacrificing accuracy.
System Model and Methodology
The authors model the system as a three-stage operation: source encoding, object multicasting, and crowd labelling.
To address the NP-hard nature of this joint optimization, the paper introduces several key strategies:
- Optimal Sequential Clustering: The authors prove that the most efficient way to group annotators is by their Signal-to-Noise Ratio (SNR) in descending order. This significantly simplifies the search space.
- JASA Algorithm: A Joint Annotator-and-Sub-channel Allocation (JASA) algorithm is developed using a branch-and-bound tree-search approach to find optimal solutions for fading channels.
- Truncated Channel Inversion (TCI): To further reduce complexity, the authors propose using power control to equalize the channels of different annotators. This transforms the problem into a 2D Knapsack Problem, solvable via Dynamic Programming in pseudo-polynomial time.
Key Insights: The Compression-Redundancy Tradeoff
A primary contribution of this work is the mathematical link established between data distortion and crowd redundancy. If an image is heavily compressed to save bandwidth, the system must "pay" for that savings by hiring more annotators to label the same image to maintain a target accuracy level.
Furthermore, the optimal strategy "flips" depending on resource scarcity:
- Bandwidth-Constrained: If spectrum is scarce, the system should maximize compression and use a larger crowd to correct the resulting errors.
- Annotator-Constrained: If human resources are scarce, the system should transmit high-quality, uncompressed data to ensure each individual label is highly reliable, thereby minimizing the need for redundant votes.
Results and Performance
Using synthetic simulations with i.i.d. Rayleigh fading channels and standard rate-distortion models, the authors demonstrated that:
- Throughput Gains: Joint allocation (JASA) significantly outperforms decoupled allocation strategies that optimize radio and human resources independently.
- Efficiency: The Dynamic Programming approach (using TCI) offers a practical alternative to the tree search, providing near-optimal performance with much lower computational overhead as the number of annotators grows.
- Optimal SNR Targeting: In the TCI model, there exists an "optimal targeted SNR." Setting this target too low wastes bandwidth, while setting it too high excludes too many capable annotators, both of which degrade total throughput.
Limitations and Future Directions
While the framework provides a robust foundation, several areas remain for future exploration. The current model assumes "flat" fading and uniform annotator skill levels. In reality, annotators vary in accuracy regardless of data quality. Additionally, the paper treats annotators as a static resource; future work could incorporate incentive mechanisms (e.g., financial rewards) to ensure participation. Other potential extensions include the use of spatial beamforming (MIMO) and more sophisticated matching algorithms between specific tasks and annotator channel conditions.
Summary updated