When tracking human movement for clinical rehabilitation, sports science, or robotics, the ultimate goal is almost always to capture the motion of the skeleton. The rigid bones and joints form the true kinematic architecture of the body. However, short of invasive surgery, sensors cannot be attached directly to bone. Instead, practitioners must rely on surface-mounted alternatives: optical markers, Inertial Measurement Units (IMUs), depth cameras, or smart textiles placed on the skin or clothing.
This constraint introduces a fundamental engineering challenge known as the Soft Tissue Artifact (STA). Because skin, muscle, and fat are non-rigid and move independently of the underlying skeleton, surface sensors capture a complex mixture of true bone kinematics and local tissue deformation. As Hu noted in an early tracking survey, high sensor precision does not guarantee high kinematic accuracy; the physical displacement of the skin creates an inherent noise floor that decouples surface measurements from internal anatomy.
The Mechanics of Surface-Internal Decoupling
To understand why the soft tissue artifact is so difficult to clear algorithmically, we must look at how it manifests across different tracking modalities. Soft tissue does not just wobble randomly; it slides, stretches, and compresses systematically based on joint angles and muscle activation.
Optical and Depth Tracking
In marker-based optical systems, the elasticity of the skin means that markers placed over anatomical landmarks shift as the joint articulates. Stillfried et al. demonstrated that this sliding behavior prevents surface markers from capturing true bone-to-bone spatial relationships. This decoupling is not limited to limbs; for example, Hashimoto et al. showed that external skin tracking fails to precisely capture the motion of deep internal structures like the larynx during swallowing.
When shifting from markers to markerless depth cameras (like the Microsoft Kinect), the problem changes but does not disappear. Instead of skin-sliding noise, computer vision models encounter systematic tracking biases and high-frequency jitter during complex or elevated joint movements, as observed by Campos et al. and Zhang et al..
Wearables and Smart Textiles
For wearable sensors like IMUs, the artifact is compounded by muscle contractions. As muscles flex and change shape, they alter the orientation and position of the sensor relative to the bone (Wu et al.). Even mechanical sensors, such as abdominal sleep-tracking belts, suffer from shifting and displacement during postural transitions (Mohammadi et al.).
[Bone Kinematics]
│
▼ (Skeletal Rotation/Translation)
[Soft Tissue / Muscle Deformation] <-- Dynamic STA Noise
│
▼ (Fabric / Garment Shifting) <-- Secondary Motion Noise
[Surface Sensor / IMU Data]
When sensors are integrated into flexible fabrics or smart sleeves, an additional layer of decoupling occurs. Zhang et al. found that the shifting of skin and muscle beneath smart textiles displaces sensors from their intended anatomical alignments. Crucially, loose garments introduce a severe geometric limitation: the fabric completely fails to track the axial rotation of the limbs (Zuo et al.). When an arm twists inside a sleeve, the fabric slides independently, blinding the sensor to that specific rotational degree of freedom. Even the physical flexing of low-cost 3D-printed wearable components under human force can introduce structural noise that corrupts joint angle readings (Xu et al.).
Establishing a True Ground Truth
To mitigate the soft tissue artifact, researchers first need a way to isolate it. However, standard motion capture systems cannot serve as a reference because they are subject to the exact same skin-movement errors. To solve this validation paradox, researchers have developed methods to "see through" or bypass soft tissue entirely.
Medical Imaging
One approach is to map internal mechanics directly using medical imaging. Stillfried et al. utilized MRI scans across dozens of static hand postures to build a high-fidelity kinematic model, bypassing surface noise to map true bone positions. Similarly, Li et al. leveraged radiographic data from clinical X-ray machines (C-arms) to track internal bone kinematics in motion, providing a baseline uncorrupted by external tissue sliding.
In-Vitro Bone Pinning
To validate wearable sensors in dynamic, high-intensity scenarios, researchers have turned to in-vitro cadaveric models. Weygers et al. created a benchmark dataset by rigidly pinning inertial sensors directly into the bones of a tibiofemoral joint. By physically eliminating the soft tissue interface, they isolated true skeletal kinematics, providing a gold-standard reference capable of separating algorithmic sensor errors from the physical noise of skin displacement.
Algorithmic Mitigation Strategies
With a clear understanding of how soft tissue behaves, modern tracking pipelines employ several mathematical and machine learning strategies to compensate for surface-internal decoupling.
Biomechanical Modeling and Interpolation
Rather than treating surface sensor data as a direct reflection of skeletal state, algorithms can treat the skin-to-bone relationship as a dynamic mapping function. Gustus et al. demonstrated that mathematical interpolation can be used to model how skin moves relative to bone during specific joint articulations, allowing software to subtract deformation noise from surface-based motion capture data.
Simulating Artifacts for Deep Learning
Many modern pose estimation networks are trained on synthetic data generated from idealized skeletal simulations. However, Kaufmann et al. and Nitschke et al. pointed out that this creates a severe domain gap: models trained on perfect bone-sensor alignments fail in the real world because they do not expect sensors to shift.
To bridge this gap, Wu et al. developed a data augmentation framework called MODA (Motion-Drift Augmentation). Instead of training networks on clean data, MODA mathematically simulates the dynamic displacement and orientation shifts caused by muscle contraction and skin deformation. By forcing the network to resolve these simulated artifacts during training, the resulting models become robust to real-world sensor-to-bone discrepancies.
Learning Secondary Motion
When dealing with the extreme decoupling of loose clothing, simple data augmentation is often insufficient. To reconstruct upper-body poses from sensors embedded in loose jackets, Zuo et al. designed a "Secondary Motion AutoEncoder." This neural network specifically models the physics of how fabric shifts against the body. By learning the statistical relationship between the garment's movement and the wearer's limbs, the system can infer hidden degrees of freedom—like axial arm rotation—even when the physical sensor on the fabric fails to register the movement directly.
Key Foundational Work
CSM-420 A Survey - Human Movement Tracking and Stroke Rehabilitation (2004) — contributed 3 concept elements to this direction.
On-body wireless inertial sensing foot control applications (2008) — contributed 2 concept elements to this direction.