Wear and Go: Calibration Free Sparse Inertial Motion Capture via Online Sensor Setup Estimation

Yuanshuo Tan, Licheng ZhongarXiv 2026 · arXiv

TL;DR

MOSAIC (Motion-based Sensor Assignment and Inertial Calibration) enables "wear and go" inertial motion capture by automatically identifying sensor placement and orientation from natural movement. By accumulating evidence over time, it eliminates the need for tedious calibration poses and allows high-fidelity 3D reconstruction using existing, frozen pose estimation models.

What Problem Does This Paper Address?

Sparse inertial motion capture (mocap) uses a handful of sensors (usually six) to reconstruct full-body movement. However, these systems typically have a high "friction" cost: the user must specify which sensor is on which limb (assignment) and perform a specific calibration pose (like a T-pose) to align the sensor's coordinate frame with the body's (calibration).

If a sensor slips or is put on slightly crooked, the reconstruction fails. While some methods attempt "online" calibration, they often require known sensor identities or specific movement sequences. The authors of this paper wanted to create a truly calibration-free system where a user can simply strap on six interchangeable sensors in any order and orientation and start moving.

Methods

The core of the paper is the MOSAIC framework, which decouples the persistent sensor setup from the ever-changing body pose.

Overview of the MOSAIC framework showing evidence encoding, session accumulation, and canonicalization for frozen pose networks.

The system works in three main stages:

  1. Window-level Evidence Extraction: A neural network processes short windows of unordered IMU data. It uses a "sensor-order-equivariant" architecture (meaning it doesn't care about the initial order of the data streams) with temporal and cross-sensor attention to predict two things: which body segment the sensor is likely on and what its mounting rotation is.
  2. Persistent Evidence Accumulation: This is the "aha" moment of the paper. Instead of relying on a single-shot guess, MOSAIC maintains a running sum of assignment scores and rotation matrices over the entire session. Because the sensor setup is fixed while the body pose changes, "ordinary" movements that might be ambiguous individually provide complementary evidence over time.
  3. Canonicalization: Once the setup is estimated (using Hungarian matching for assignment and SVD-based projection for rotation), MOSAIC transforms the raw IMU streams into a "canonical" format. This allows the data to be fed into frozen, pretrained pose estimators (like LSTM, PIP, or DynaIP) without needing to retrain them for uncalibrated data.

Datasets

The authors used a mix of synthetic and real-world data:

  • AMASS: A large-scale database of human motion used to generate synthetic IMU measurements for pretraining.
  • DIP-IMU: A standard dataset for sparse inertial mocap. Subjects 1–8 were used for fine-tuning, while 9–10 were held out for evaluation.
  • TotalCapture: Used as an additional evaluation benchmark to test the system's generalizability.
  • Live Demo Data: The authors recorded their own dataset using Xsens IMUs with randomized setups and a RealSense camera for visual ground truth.

Key Insights and Ideas

  • Persistence is Key: The fundamental insight is that the sensor-to-segment relationship is a persistent physical parameter. While a single arm swing might not reveal the leg sensor's orientation, a few seconds of walking and moving naturally eventually constrains all degrees of freedom.
  • Modular Interface: By treating setup estimation as a "pre-processing" step that canonicalizes data, MOSAIC acts as a universal adapter. This means researchers can improve pose estimation models and setup estimation models independently.
  • Evidence Pooling: The use of Matrix Fisher distributions motivates their choice to sum evidence matrices. This mathematical framework allows them to treat overlapping windows of motion as additive evidence rather than independent observations.

Key Results

  • Rapid Recovery: In evaluations, MOSAIC recovered correct sensor assignments within 2 to 8 seconds of ordinary motion.
  • Near-Calibrated Accuracy: For sensors with bounded mounting offsets (up to 30°), the system achieved pose accuracy nearly identical to systems that were manually calibrated (within ~1° of angular error).
  • Robustness to Arbitrary Orientation: Even when sensors were attached with completely random rotations (the "Arbitrary" regime), MOSAIC's accumulation strategy significantly outperformed baselines, reducing error by over 20% compared to single-window estimates.
  • Plug-and-Play: The authors demonstrated that MOSAIC successfully "canonicalized" inputs for multiple state-of-the-art pose networks (PIP, DynaIP) without any modifications to those networks.
Figure 1. MOSAIC enables calibration free inertial motion capture. Users attach six interchangeable IMUs to prescribed body sites without specifying sensor identities or arbitrary mounting orientations and begin moving without a calibration pose. MOSAIC accumulates evidence from ordinary motion to recover the persistent sensor setup, progressively aligning the measurements for accurate reconstruction with frozen pose networks. Project page: https://tanyuanshuo.github.io/wearandgo_page/

Limitations and Future Work

While MOSAIC is a major step toward consumer-ready mocap, some limitations remain:

  • Prescribed Sites: The system currently assumes sensors are placed at six specific "standard" sites (forearms, shanks, head, pelvis). It cannot yet handle arbitrary placement (e.g., a sensor on the chest instead of the head).
  • Rigid Attachment: The model assumes the sensor doesn't move relative to the bone once attached. It does not explicitly model "soft tissue artifact" or sensors sliding down a limb during a long session.
  • Future Directions: The authors suggest that future work could explore more flexible sensor layouts and methods to handle time-varying alignment caused by sensor slippage.

Summary updated

Abstract

Sparse inertial motion capture typically assumes that each sensor's anatomical identity and mounting orientation are known. We present MOSAIC (Motion-based Sensor Assignment and Inertial Calibration), a framework that recovers this setup from ordinary motion without a calibration pose. Our central insight is that body pose changes continuously while the sensor setup persists, allowing movements that are individually ambiguous to provide complementary evidence over time. A setup network processes unordered sensor streams with temporal and cross-sensor attention, then accumulates assignment and rotation evidence into a session-level estimate. The recovered setup canonicalizes measurements for frozen pretrained pose estimators, separating sensor configuration from motion reconstruction. For sensors worn at prescribed body sites, evaluation on real inertial recordings with randomized setups shows near-calibrated pose accuracy for bounded offsets and substantial benefits from accumulation under arbitrary orientations. Transfer across pose networks supports setup recovery as a reusable interface between interchangeable sensors and pretrained pose models.
BibTeX
@misc{tan2026wear,
  author = {Yuanshuo Tan and Licheng Zhong},
  title = {{Wear and Go: Calibration Free Sparse Inertial Motion Capture via Online Sensor Setup Estimation}},
  year = {2026},
  eprint = {2611.22894},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.22894},
}
Figure