RetailOpt: An Opt-In, Easy-to-Deploy Trajectory Estimation System Leveraging Smartphone Motion Data and Retail Facility Information

Ryo Yonetani, Jun Baba, Yasutaka Furukawa

Summary

RetailOpt is a novel indoor tracking system that estimates customer trajectories using only existing data: smartphone motion sensors, store maps, and purchase records. By treating purchased items as "spatial anchors" and using a combination of neural continuous optimization and Viterbi-based discrete optimization, it achieves high-accuracy localization without requiring expensive hardware like beacons or privacy-invasive surveillance cameras.

The Challenge of Indoor Analytics

Accurately tracking how customers move through a retail space is the "holy grail" for brick-and-mortar analytics, enabling targeted marketing, better inventory management, and improved store layouts. However, existing solutions face a "trilemma" of cost, maintenance, and privacy:

  1. Infrastructure-heavy systems (like Ultra-Wideband or Bluetooth beacons) are expensive to install and calibrate.
  2. Vision-based systems (cameras) raise significant privacy concerns and often lack explicit user consent.
  3. Fingerprinting methods (Wi-Fi or magnetic fields) require labor-intensive "site surveys" that must be redone whenever the environment changes.

RetailOpt addresses these challenges by providing an opt-in, zero-hardware-deployment system that respects privacy while maintaining high accuracy in complex indoor environments.

The RetailOpt Approach

RetailOpt follows a multi-stage pipeline to turn noisy sensor data into a clean trajectory:

  1. Inertial Navigation: The system uses the smartphone’s IMU (accelerometer and gyroscope) to estimate a relative trajectory. They employ RoNIN, a deep-learning-based inertial odometry model, to handle the noise and drift inherent in handheld devices.
RetailOpt transforms noisy inertial data into optimized trajectories.
  1. Anchor Extraction: The system extracts two types of anchors from retail facility data. Time-known anchors, such as POS checkout events, provide a location and a precise timestamp. Time-unknown anchors are derived from digital receipts; if a customer bought apples, the system knows they visited the "apple shelf" at some point, but not exactly when.
  2. Continuous Optimization: A neural network utilizing an attention mechanism learns to warp and shift the noisy relative trajectory to align with these anchors. Because the visit times for most anchors are unknown, the authors use a "soft-matching" loss function that probabilistically associates trajectory points with physical shelf locations.
  3. Discrete Optimization: To ensure the final path respects physical boundaries, the system projects the transformed trajectory onto a searchable graph of the store's walkable area. A Viterbi algorithm then finds the most likely valid path that stays close to the optimized trajectory while avoiding obstacles like walls or shelves.

Key Innovations

Purchase Records as Semantic Cues

The most innovative idea is treating a shopping list as a set of spatial constraints. By cross-referencing a receipt with a store map, the "what" of a purchase becomes the "where" of a trajectory.

Using purchase records as spatial anchors to refine trajectory estimation.

Handling Temporal Uncertainty

Traditional point-set registration struggles when the visit time is unknown. RetailOpt’s "soft-matching" approach allows the optimization to discover the most likely visit times automatically. Furthermore, using an attention layer in the transformation network allows the system to handle a variable number of anchors and different journey lengths seamlessly.

Experimental Results

The authors evaluated RetailOpt using the RetailOpt Dataset (15 participants in a 100 m² store) and the public Inertial Localization Dataset (NILoc).

  • Superior Accuracy: In the store environment, RetailOpt achieved an average positional error of 0.82m to 1.25m (depending on phone placement), significantly outperforming baseline methods like TSP solvers (~3m error) and standard RoNIN (~3m+ error).
  • Robustness to Drift: Even on long trajectories up to 400m, RetailOpt maintained much lower error rates than state-of-the-art inertial models, which often require extensive site-specific training.
  • Efficiency: The system is computationally efficient, processing a 10-minute trajectory in less than 2 seconds on a standard GPU.

Limitations and Future Work

While promising, the system faces challenges when a store stocks the same item in multiple locations (the "Multiple Shelf" problem). Additionally, accuracy degrades when the phone is loose in a bag, as the IMU data becomes detached from the person's actual steps. Future work will focus on resolving these ambiguities and exploring cross-retailer tracking frameworks.

Summary updated

BibTeX
@inproceedings{yonetani2024retailopt,
  author = {Ryo Yonetani and Jun Baba and Yasutaka Furukawa},
  title = {{RetailOpt: An Opt-In, Easy-to-Deploy Trajectory Estimation System Leveraging Smartphone Motion Data and Retail Facility Information}},
  booktitle = {ISWC},
  year = {2024},
  doi = {10.1145/3675095.3676623},
}
Figure