TL;DR
This paper presents a novel system for real-time, 3D localization between two smartphones using only their built-in microphones, speakers, accelerometers, and compasses. The system achieves centimeter-level accuracy, enabling new classes of "high-speed, locational, phone-to-phone" (HLPP) applications like interactive mobile games without external infrastructure.
What Problem Does This Paper Address?
The proliferation of smartphones has opened doors to many innovative applications, but certain types of interactive experiences remain elusive. Specifically, applications that require precise, real-time 3D positioning between multiple mobile devices without relying on external infrastructure (like Wi-Fi access points or GPS) are currently unsupported. Examples include multiplayer games where phones act as game controllers or where multiple phone screens combine into a larger display. The core challenge is to enable two phones to accurately determine their relative 3D positions in real-time, even as they move, using only the sensors commonly found on modern smartphones. Existing solutions often require dedicated hardware or external infrastructure, making them unsuitable for ad-hoc, mobile scenarios.
Methods
The proposed solution leverages a combination of acoustic signals and inertial sensors. The core idea is to use acoustic cues, specifically Time-of-Arrival (TOA) and signal power differences, captured by the multiple microphones on each phone.
- Acoustic Localization: The system builds upon the BeepBeep protocol for 1D distance estimation. By using two microphones on each phone, they can derive 2D angle information. The combination of TOA and power difference cues helps resolve ambiguities and improve angle estimates.
- Sensor Fusion: Accelerometers and digital compasses are crucial for resolving ambiguities in acoustic-only localization and for tracking movement when phones are outside the optimal "alignment region" (where mics and speakers are facing each other).
- Kalman Filtering: A loose time synchronization protocol and an Extended Kalman Filter are employed to achieve continuous localization. The Kalman filter smooths noisy measurements from the sensors, tracks the phones' relative positions and velocities, and accounts for various measurement errors.
- Fallback Mode: To handle situations where acoustic localization is less reliable (e.g., phones not facing each other), a fallback mode uses IMU (Inertial Measurement Unit) data from accelerometers and compasses for displacement tracking. The system dynamically switches between primary (acoustic) and fallback (IMU) modes based on estimated error.
Datasets
Not explicitly discussed in the paper. The evaluation uses custom experimental setups with off-the-shelf Nexus One phones.
Key Insights and Ideas
- Leveraging Commodity Hardware: The paper demonstrates that centimeter-level 3D localization is feasible using only the sensors already present in modern smartphones, without requiring specialized hardware.
- Acoustic Cues are Powerful: TOA and signal power differences, when combined and processed with sensor fusion, provide surprisingly accurate localization data.
- Sensor Fusion is Key: The synergy between acoustic sensors and inertial sensors (accelerometer, compass) is critical for overcoming the limitations of each individual sensor type and achieving robust 3D localization.
- Kalman Filtering for Continuity: The use of an Extended Kalman Filter is essential for smoothing noisy measurements and providing continuous, real-time position estimates, which is vital for interactive applications.
- Dynamic Mode Switching: The fallback mode and the mechanism for switching between primary and fallback modes are crucial for maintaining localization accuracy across a wider range of phone orientations and movements.
Key Results
The experimental results are promising:
- Accuracy: The system achieves localization resolution within 13.9 cm for 90% of estimates and within 4.9 cm for 50% of estimates when phones are several meters apart.
- Motion Tolerance: The system operates smoothly with simulated motion paths up to distances of 2 meters.
- Cue Combination: Combining TOA and power cues leads to significantly better accuracy than using TOA alone.
- Kalman Filter Benefits: The Kalman filtered estimates show substantial improvement over raw TOA and TOA+Power estimates, reducing average position error.
Limitations and Future Work
- Environmental Sensitivity: The system is susceptible to errors from solid-body interference and variations in sound propagation speed due to humidity. It is not ideal for environments with many obstructions.
- Background Noise: The impact of background noise on cue accuracy has not been fully evaluated.
- Motion Path Complexity: While the system handles simulated motion well, sharp turns and sudden acceleration changes can be difficult to compensate for. Future work includes detecting such changes and adapting the Kalman filter accordingly.
- Hardware Limitations: The current implementation is constrained by the Android OS's inability to simultaneously access two microphones. Future work could explore OS or driver modifications for improved performance.
- Calibration: The paper notes that the system relies on empirical measurements for cue weighting, and that more powerful devices with additional sensors could potentially achieve even better accuracy.
- Ground Truth Acquisition: Establishing precise ground truth for motion paths was challenging; vision-based tracking is suggested as a future improvement.
Summary updated