When tracking human or animal movement, a single sensor modality often leaves critical physical ambiguities unresolved. Consider an accelerometer attached to a foot: it can measure the rapid deceleration of a step, but it cannot easily distinguish whether that foot has struck a hard concrete floor, landed on a soft mat, or merely reached the end of an airborne kick. Similarly, a pressure sensor under the sole can tell you exactly when a foot is on the ground, but it becomes completely blind the moment the foot lifts into the air.
To overcome these limitations, modern wearable systems rely on multimodal sensor fusion. By mathematically combining kinematic data (the geometry and description of motion, such as acceleration and angular velocity) with kinetic data (the forces causing motion, such as ground reaction forces and plantar pressure), these systems build a holistic, unambiguous representation of physical activity.
Resolving Rotational Ambiguities in Kinematics
Before bridging motion and force, systems must first resolve ambiguities within kinematics itself. Linear accelerometers are notoriously prone to orientation drift and struggle to separate the acceleration of gravity from the actual movement of the subject.
To solve this, researchers combine triaxial accelerometers with gyroscopes, which measure angular velocity. This combination allows algorithms to continuously track the sensor's orientation and mathematically decouple rotational maneuvers from linear displacement.
Alessandro Ghio et al. demonstrated the power of this pairing in A Public Domain Dataset for Human Activity Recognition using Smartphones, showing that fusing acceleration and angular velocity provides a massive performance boost in human activity recognition (HAR) compared to using acceleration alone. The reason is intuitive: complex human movements often share identical linear acceleration patterns but feature entirely different rotational signatures.
For instance, Heike Leutheuser et al. showed in Hierarchical, Multi-Sensor Based Classification of Daily Life Activities: Comparison with State-of-the-Art Algorithms Using a Benchmark Dataset that activities like climbing stairs or performing manual labor become reliably distinguishable only when rotational components are captured. This principle extends beyond human biomechanics; in marine biology, T. Aran Mooney et al. utilized this exact fusion in Augmenting biologging with supervised machine learning to study in situ behavior of the medusa Chrysaora fuscescens to differentiate between the active swimming and passive drifting states of jellyfish in the open ocean.
To further refine this kinematic picture, modern systems incorporate magnetometers and atmospheric pressure sensors to anchor absolute orientation and altitude changes. This approach was leveraged by Yeji Woo et al. in Human Activity Recognition Dataset for Pedestrians with Mobility Disabilities to track individuals using assistive devices, and by Fabian Clemens Weigend et al. in Anytime, Anywhere: Human Arm Pose from Smartwatch Data for Ubiquitous Robot Control and Teleoperation to accurately reconstruct 3D arm poses using only a standard smartwatch.
Bridging Kinematics and Kinetics
While a perfected kinematic model tracks where a limb is moving, it still lacks an understanding of environmental interaction forces. Fusing Inertial Measurement Units (IMUs) with force or pressure sensors bridges this gap, mapping the relationship between a body's motion and the constraints of its environment.
+-----------------------+ +-----------------------+
| Kinematic Sensors | | Kinetic Sensors |
| (IMUs: Accel + Gyro) | | (Pressure/Force Pads) |
+-----------+-----------+ +-----------+-----------+
| |
| Trajectory & | Timing, Contact &
| Orientation | Weight Distribution
+--------------+---------------+
|
v
+------------------------------+
| Multimodal Fusion Layer |
+--------------+---------------+
|
v
+------------------------------+
| Unambiguous State Estimate |
| (e.g., Gait Phase, Posture) |
+------------------------------+
Marco Benocci et al. pioneered this integration in A wireless system for gait and posture analysis based on pressure insoles and Inertial Measurement Units. By combining high-density pressure-sensing insoles with multi-axis IMUs, the authors simultaneously captured foot-ground contact forces and limb kinematics. This allowed them to precisely identify complex gait phases, such as the transition between single and double support, which cannot be reliably detected by either sensor type alone.
This real-time synchronization is critical for closed-loop medical interventions. For example, in assistive robotics, Ling-Fung Yeung et al. utilized force-sensing resistors alongside shank-mounted IMUs in Randomized controlled trial of robot-assisted gait training with dorsiflexion assistance on chronic stroke patients wearing ankle-foot-orthosis and Effects of wearable ankle robotics for stair and over-ground training on sub-acute stroke: a randomized controlled trial. By fusing these modalities, the robotic orthosis can immediately distinguish between level-ground walking and stair navigation, dynamically adjusting its mechanical assistance to match the user's intent and environmental context.
Similarly, Joonbum Bae developed a tele-monitoring system for gait rehabilitation in A tele-monitoring system for gait rehabilitation with an inertial measurement unit and a shoe-type ground reaction force sensor that pairs an IMU with a shoe-based ground reaction force (GRF) sensor. This synergistic fusion provides a richer dataset for remote physical therapy than traditional laboratory setups.
Interestingly, the mathematical relationship between these two domains flows both ways. While physical force sensors provide direct measurements, Marco Paolieri et al. demonstrated in Estimating Ground Reaction Forces From Inertial Sensors that by combining acceleration and angular velocity measurements through machine learning, it is possible to accurately estimate ground reaction forces even when physical pressure sensors are entirely absent.
Holistic State Estimation
When scaled up from individual limbs to the entire body, multimodal fusion enables comprehensive ergonomic and behavioral tracking.
In industrial settings, Eya Barkallah et al. combined an IMU embedded in a worker's helmet with force sensors in their footwear in Wearable Devices for Classification of Inadequate Posture at Work Using Neural Networks. This configuration captures head movement and foot pressure distribution simultaneously, allowing neural networks to identify inadequate or dangerous postures that could lead to musculoskeletal disorders.
The paradigm also applies to fine-grained manipulation. Yaxu Xue et al. developed a hand motion recognition system in Multiple Sensors Based Hand Motion Recognition Using Adaptive Directed Acyclic Graph that integrates electromyography (EMG) sensors, a data glove, and tactile force sensors. By capturing muscle activity (intent), finger joint angles (kinematics), and contact forces (kinetics) simultaneously, the system creates an exhaustive representation of complex hand grasps.
As highlighted by Zhiyuan Zhang et al. in Insole Systems for Disease Diagnosis and Rehabilitation: A Review and validated by Faisal Arafsha et al. in Instrumented Wireless SmartInsole System for Mobile Gait Analysis: A Validation Pilot Study with Tekscan Strideway, moving beyond isolated sensing to a unified kinematic-kinetic framework is what ultimately allows clinical-grade diagnostics and rehabilitation monitoring to transition out of specialized laboratories and into everyday life.
Key Foundational Work
A Public Domain Dataset for Human Activity Recognition using Smartphones (2013) — contributed 2 concept elements to this direction.
A wireless system for gait and posture analysis based on pressure insoles and Inertial Measurement Units (2009) — contributed 2 concept elements to this direction.
A tele-monitoring system for gait rehabilitation with an inertial measurement unit and a shoe-type ground reaction force sensor (2013) — contributed 2 concept elements to this direction.