Using Acoustic Signal and Image to Achieve Accurate Indoor Localization

Rui Xi, Daibo Liu, Mengshu Hou, Yujun Li, Jun Li2018 · DOI

TL;DR

This paper introduces SITE, a novel indoor localization system that fuses acoustic signals and smartphone camera images to achieve accurate and robust positioning. By combining Doppler effect-based acoustic direction finding with visual structure-from-motion (VSFM), SITE overcomes the limitations of individual modalities, offering a practical solution for ubiquitous indoor location services.

What Problem Does This Paper Address?

Accurate and inexpensive indoor localization remains a significant challenge. Existing methods often fall into two categories: fingerprinting-based, which require time-consuming site surveys and frequent database updates, and ranging-based, which typically demand specialized, expensive hardware. While acoustic localization offers a low-cost, off-the-shelf solution, it suffers from robustness issues due to environmental interference. Similarly, image-based localization can be accurate but often requires significant computational resources or specific environmental features. This paper aims to bridge this gap by developing a system that leverages the strengths of both acoustic and visual data to provide a more accurate, robust, and practical indoor localization solution.

Methods

SITE employs a multi-stage approach:

  • Acoustic Localization: This module uses the Doppler effect from pre-deployed acoustic anchors (emitting inaudible frequencies) to estimate the relative direction between the smartphone and each anchor. By analyzing the frequency shift of the received signals, the system can determine the phone's velocity and displacement. A key innovation here is the use of a "Decision Scheme" that assesses the reliability of initial location estimates derived from acoustic data. This scheme identifies "ConvergenceSets" – subsets of acoustic anchor readings that consistently point to a location within a certain standard deviation threshold.
  • Decision Scheme: This module is crucial for filtering out unreliable acoustic-based location estimates. It analyzes the standard deviation (SD) of multiple candidate locations derived from different subsets of acoustic anchors. If the SD is below a predefined threshold (δ = 0.3m), the locations are considered "CONVERGED" and likely accurate. If not, or if multiple converged sets exist, the system proceeds to the next stage.
  • Position Refinement (VSFM): If the acoustic localization alone is deemed insufficient (i.e., the Decision Scheme returns "DIVERGED"), SITE utilizes the smartphone's camera. It employs Visual Structure-from-Motion (VSFM) to reconstruct a 3D scene and derive relative camera poses and locations from a series of captured images. This visual information is then fused with the initial acoustic-based location estimate to refine the user's position. An optimal transformation relationship is found between initial acoustic locations and VSFM-derived relative coordinates to achieve the final, more accurate location.

Datasets

Not explicitly discussed in the paper. The authors mention conducting experiments in a "building lobby" and a "university library" using a custom-deployed platform of acoustic sources and testing spots.

Key Insights and Ideas

  • Synergy of Acoustic and Visual Data: The core insight is that combining acoustic direction finding with visual information (VSFM) creates a more robust and accurate localization system than either modality alone. Acoustic signals provide a low-cost, ubiquitous sensing mechanism, while VSFM offers precise relative pose estimation.
  • The "Decision Scheme" for Robustness: The paper highlights the importance of intelligently filtering acoustic localization results. The "ConvergenceSet" concept, based on the standard deviation of candidate locations, effectively identifies reliable estimates and avoids using potentially erroneous ones, significantly improving robustness.
  • Observation on Location Diversity: A key observation is that when multiple acoustic anchors are used, the estimated locations, even if scattered, tend to converge towards the true location if they are reliable. The degree of this convergence (measured by standard deviation) is a strong indicator of localization accuracy.
  • VSFM for Refinement: Using VSFM to refine initial acoustic estimates is a practical approach to overcome the inherent noise and interference in indoor acoustic environments, especially when the acoustic data alone is insufficient.

Key Results

  • SITE achieves a median localization error of 0.42 m in Non-line-of-sight (NLOS) conditions and 0.39 m in Line-of-sight (LOS) conditions in experimental evaluations.
  • Compared to the state-of-the-art Swadloon system, SITE demonstrates a median improvement in localization accuracy of 54.67% in LOS and 43.83% in NLOS.
  • The system shows good performance in both a building lobby and a more challenging library environment, outperforming Swadloon in both scenarios.
  • The paper also analyzes the impact of the number of acoustic sources, showing that accuracy improves with more sources, but also increases computational workload.

Limitations and Future Work

  • 3D Localization: SITE currently assumes a 2D plane and that the phone and acoustic anchors are at approximately the same height. It cannot estimate 3D coordinates directly.
  • Semantic Information: The current localization results are numerical coordinates. The authors suggest that associating these coordinates with semantic information (e.g., room names, functions) would be more valuable for users.
  • Computational Overhead: While manageable, the acoustic signal processing and VSFM can still be computationally intensive for smartphones. Future work could explore offloading computation to a backend server.
  • Environmental Dependence: The performance can be affected by factors like sound pollution and the layout of acoustic anchors.
  • Future Work: The authors suggest generating semantic maps using crowdsourced data and associating environmental information with coordinates to provide richer context. They also mention exploring dead reckoning and particle filtering for trajectory tracking.

Summary updated

BibTeX
@misc{xi2018using,
  author = {Rui Xi and Daibo Liu and Mengshu Hou and Yujun Li and Jun Li},
  title = {{Using Acoustic Signal and Image to Achieve Accurate Indoor Localization}},
  year = {2018},
  doi = {10.3390/s18082566},
}

Loading related content...

Figure