Continuous-Time Set-Transformers for Asynchronous WiFi-IMU Indoor Localization

Mohamed Bachar, Ilyass Abouelaziz, Yuehua DingarXiv 2026 · arXiv

Indoor localization is a notoriously "messy" data problem. While GPS works wonders outdoors, indoor environments force us to rely on a cocktail of sensors—primarily WiFi and Inertial Measurement Units (IMUs). The headache? These sensors speak different languages and operate on different schedules.

In their recent paper, "Continuous-Time Set-Transformers for Asynchronous WiFi-IMU Indoor Localization," Mohamed Bachar and his team propose a elegant solution: stop trying to force these sensors onto a shared clock and start treating them as an unordered set of timestamped tokens.

TL;DR

The authors present a continuous-time Set-Transformer that fuses asynchronous WiFi and IMU data without the need for resampling or fixed time grids. By representing sensor readings as timestamped tokens in a shared latent space, the model achieves robust localization that handles missing or stale data far better than traditional "branch-per-modality" architectures.

What Problem Does This Paper Address?

Most indoor localization systems face a fundamental synchronization challenge. WiFi scans provide absolute reference points but arrive sparsely and irregularly. IMUs provide dense, high-frequency relative motion data but drift over time.

Existing methods typically try to "fix" this by resampling all data onto a common time grid or using separate neural network branches for each sensor. This is brittle: if a WiFi scan is delayed or an IMU window is missing, the fixed architecture often breaks or requires manual "null vector" padding. The authors aim to create a single, unified model that is natively invariant to the timing and order of incoming sensor data.

Methods

The core innovation is the Continuous-Time Set-Transformer. Instead of a sequence, the input is an unordered set of observations.

  1. Universal Tokenization: Every sensor reading (a WiFi scan or an IMU window) is converted into a $D$-dimensional token. This token combines three things: the sensor measurement, a learned modality embedding (to identify if it's WiFi or IMU), and a continuous-time encoding.
  2. Continuous-Time Encoding: Rather than using discrete position indices (like standard Transformers), the model uses sinusoidal functions of the real-valued elapsed time ($\Delta t$) since the observation. This allows the model to understand exactly how "stale" a piece of data is without resampling.
  3. Specialized Encoders:
    • WiFi-Net: Scores RSSI scans against learned "anchor" points to create a spatial fingerprint.
  4. IMU-CNN: Uses 1D convolutions to extract motion features from a window of inertial data.
  5. Set-Level Fusion: A single self-attention block processes the entire set of tokens. Because self-attention is permutation-invariant, the model doesn't care about the order of inputs—only their content and timestamps.
  6. Robustness Training: The authors used "modality dropout" during training, randomly masking entire sensors or individual observations. This teaches the model to degrade gracefully when a sensor fails.
Figure 1: Overview of the proposed continuous-time transformer: each observation becomes one token, a single self-attention block fuses the set, and a learnable position query reads out (ˆx, ˆy).

Datasets

The model was evaluated across five different contexts to test both individual encoders and the fused system:

  • Webots TIAGO++: A simulation environment for controlled testing.
  • MSILN (Microsoft Indoor Localization): A challenging real-world dataset used for "cross-session" testing (training on one day, testing on another).
  • IMUWiFine: A real-world dataset for within-session testing.
  • UJIIndoorLoc & RoNIN: Standard benchmarks used to validate the WiFi and IMU encoders in isolation.

Key Insights and Ideas

  • Permutation Invariance is a Feature: By treating observations as a set rather than a sequence, the model naturally handles multi-rate sensors. A WiFi token can "attend" to an IMU token from a different instant just as easily as one from its own.
  • Time as a Continuous Variable: Using sinusoidal encodings for $\Delta t$ (similar to Time2Vec) removes the need for expensive ODE solvers or the inaccuracies of binning time into discrete slots.
  • Learned Robustness: The "modality dropout" strategy ensures the model doesn't over-rely on a single stream. If WiFi disappears, the model relies more on the IMU's dead-reckoning capabilities.
  • Efficiency: Despite its flexibility, the model is fast. It runs at ~4.77 ms per sample on a mid-range GPU (Quadro P4000), making it viable for real-time use.
Figure 2: WiFi-Net encoder. An RSSI scan is scored against Na learned reference points; a softmax over the similarities gives attention weights that aggregate the learned embeddings of those reference points into a token, refined by a residual MLP head.

Key Results

The "headline" result comes from the MSILN cross-session test, which simulates the drift and environmental changes of a real deployment:

  • Accuracy: The proposed model achieved a 10.90m Mean Absolute Error (MAE) on the test set. For comparison, a standard WiFi fingerprinting baseline (wlan_localization) lagged behind at 28.31m, and a learned LSTM fusion (IMUWiFine) struggled significantly at 52.69m.
  • Graceful Degradation: In ablation studies, the model maintained reasonable accuracy even when one modality was completely suppressed. With WiFi only, MAE was 0.86m (in sim); with IMU only, it rose to 3.50m. Fusing both brought it down to 0.61m.
  • Comparison: As shown in the table below, the "Ours" approach is the only one to satisfy all three desired properties: continuous-time tokens (P1), single-block cross-modal fusion (P2), and learned robustness (P3).
Table 1: Methods scored against the three proper- ties. P1: continuous-time tokens without resampling or an ODE solver. P2: one permutation-invariant set transformer for cross-modal and cross-time fusion in a single self-attention block. P3: learned robustness to missing or stale sensors, validated across sessions. ✓holds, ∼partial, – absent or not applicable.

Limitations and Future Work

  • IMU Performance Gap: While the IMU encoder is lightweight (0.05M parameters), it still underperforms much larger, specialized models like ResNet1D (4.6M parameters) on pure inertial tracking.
  • Synthetic WiFi: The sub-meter accuracy in simulation is likely optimistic, as the simulated WiFi field is much "cleaner" than real-world radio environments.
  • Future Directions: The authors suggest integrating more modalities (like light or pressure sensors) and incorporating physical constraints (like wall maps) directly into the fusion mechanism to further reduce drift.

Summary updated

Abstract

Indoor localization commonly combines WiFi received signal strength indicator (RSSI) fingerprints with inertial measurements. However, these sensor streams are sampled asynchronously at different rates; therefore, observations rarely align in time and may be missing or stale. Most existing methods resample the data onto a shared clock and process each modality using a separate branch. We instead present a continuous-time transformer that represents observations as an unordered set of timestamped tokens. Each token encodes its measurement, modality, and elapsed time since acquisition. A single attention block jointly fuses information across modalities and time, avoiding both resampling and modality-specific branches while remaining invariant to token order. During training, we randomly mask entire modalities and individual observations, enabling the model to operate under missing or outdated inputs. We validate the modality-specific encoders on their respective benchmarks and evaluate the fused model on both simulated and real datasets, including a crosssession dataset collected on unseen days. The proposed approach improves upon an open WiFi fingerprinting baseline under cross-session drift, remains competitive with learned fusion and dead-reckoning baselines, and degrades gracefully as observations become missing or stale. These results suggest that a single continuoustime transformer provides a practical framework for indoor localization from asynchronous sensors.
BibTeX
@misc{bachar2026continuous,
  author = {Mohamed Bachar and Ilyass Abouelaziz and Yuehua Ding},
  title = {{Continuous-Time Set-Transformers for Asynchronous WiFi-IMU Indoor Localization}},
  year = {2026},
  eprint = {2611.13057},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.13057},
}
Figure