Abstract
Inspired by the recent success of RGB-D cameras, we propose the enrichment of RGB data with an additional "quasi-free" modality, namely, the wireless signal (e.g., wifi or Bluetooth) emitted by individuals' cell phones, referred to as RGB-W. The received signal strength acts as a rough proxy for depth and a reliable cue on their identity. Although the measured signals are highly noisy (more than 2m average localization error), we demonstrate that the combination of visual and wireless data significantly improves the localization accuracy. We introduce a novel image-driven representation of wireless data which embeds all received signals onto a single image. We then indicate the ability of this additional data to (i) locate persons within a sparsity-driven framework and to (ii) track individuals with a new confidence measure on the data association problem. Our solution outperforms existing localization methods by a significant margin. It can be applied to the millions of currently installed RGB cameras to better analyze human behavior and offer the next generation of high-accuracy location-based services.
Introduction
Tracking individuals in crowded public spaces using a single RGB camera is a fundamental challenge in computer vision. Two primary obstacles hinder performance: depth ambiguity, where the distance of a person from the camera is difficult to estimate from a 2D projection, and frequent occlusions that lead to identity swaps or lost tracks. While RGB-D sensors (like the Kinect) provide depth, their high cost and limited range make them impractical for large-scale deployments in public infrastructure.
This research introduces RGB-W, a multimodal approach that enriches standard video data with "quasi-free" wireless signals (Wi-Fi or Bluetooth) emitted by smartphones. By leveraging the unique identity and signal strength (RSS) of these devices, the system creates a robust tracking framework that maintains identity even under total visual occlusion. Since over 50% of people in public spaces typically have wireless enabled, this data provides a powerful, hardware-free secondary modality.
Methodology: From RSS to Ring Images
The core challenge in fusing wireless and visual data is the high level of noise in RSS readings, which typically results in localization errors exceeding 2 meters due to multipath effects and signal absorption. The authors propose a novel representation called Ring Images to bridge the gap between 1D signal strength and 2D image data.
Instead of treating a wireless signal as a single point, it is modeled as a probability distribution over the ground plane. Using power regression and Support Vector Machines (SVM), the system filters noisy readings and defines error bounds via Support Vector Regression (SVR), creating "rings" where a person is likely to be located.
These rings are integrated into a sparsity-driven optimization framework. The localization problem is formulated as a sparse reconstruction task ($Ax = b$), where the ground plane is discretized into a grid. A "Cascade of Solvers" is employed:
- Lasso: Utilizes the known number of unique MAC IDs as a constraint to locate individuals carrying active wireless devices.
- Basis Pursuit De-Noising (BPDN): Analyzes the visual residual to detect individuals without wireless signals, ensuring the system remains effective for the entire crowd.
Experimental Results
To evaluate the proposed method, the authors introduced a new RGB-W dataset featuring high-density indoor and outdoor scenes (up to 1 person per $m^2$). The fusion of modalities yielded significant improvements:
- Localization Performance: The RGB-W model achieved a 69.5% recall, a substantial increase over the 45.6% achieved by the best vision-only baseline (O-Lasso).
- Error Reduction: The noise classification pipeline successfully reduced average wireless localization error from over 2 meters to 0.81 meters.
- Identity Persistence: The unique nature of MAC IDs acted as an "anchor," allowing the system to maintain tracks through long-term occlusions where visual-only methods typically fail.
Conclusion and Future Work
RGB-W demonstrates that ubiquitous wireless signals can serve as a powerful proxy for depth and identity in human tracking. By representing noisy signals as ring images and applying sparsity constraints, the framework significantly outperforms traditional monocular vision systems.
While the system is robust, future work will address extreme signal fluctuations and privacy considerations, ensuring visitors can opt-in to such services. The authors suggest that more complex temporal modeling—comparing the movement dynamics of wireless signals versus visual tracklets—could further refine data association in extremely dense crowds. This approach can be retrofitted to existing camera infrastructures to enable the next generation of high-accuracy location-based services.
Summary updated