TL;DR
This paper introduces a highly efficient, tiny residual neural network capable of recognizing gym workouts directly on low-power wearable devices. By leveraging a novel human body capacitance sensing method and optimizing for resource-constrained microcontrollers, they achieve near real-time, on-device activity recognition with minimal accuracy loss, paving the way for more intelligent and autonomous fitness trackers.
What Problem Does This Paper Address?
Current smart wearables offer basic fitness tracking but struggle with recognizing a wide variety of gym exercises automatically. This often requires manual input from the user, limiting the user experience and the device's utility. The core challenge lies in the significant computational demands of sophisticated activity recognition models, which are typically too power-hungry and resource-intensive for small, battery-operated wearable devices. Existing solutions often rely on cloud processing, introducing latency and privacy concerns, or use non-wearable sensors that can be inconvenient. This paper aims to bridge this gap by enabling accurate, real-time gym workout recognition directly on the edge, on the wearable device itself.
Methods
The authors developed a lightweight residual neural network (ResNet) architecture specifically designed for embedded systems. This model is built upon a 1D convolutional layer followed by three identity blocks, each containing three 1D convolutional layers. This architecture is chosen for its ability to extract features from time-series data effectively.
A key innovation is the use of a human body capacitance (HBC) sensing unit in addition to traditional Inertial Measurement Units (IMUs). HBC sensing captures variations in capacitance between the body and its environment, which can be indicative of limb movements. This dual-sensing approach provides richer data for activity recognition.
To deploy this model on resource-constrained devices, the researchers employed quantization, specifically 8-bit quantization for weights and activations. This process significantly reduces the model's memory footprint and computational requirements with minimal impact on accuracy. They also explored model compression techniques like quantization, which are crucial for edge AI.
Datasets
The paper introduces a newly collected gym workout dataset comprising fifty sessions from ten volunteers performing eleven different gym exercises. This dataset includes data from IMU sensors (accelerometer and gyroscope) and the novel HBC sensing unit, collected from the wrist. The data was sampled at a frequency of 20 Hz. The dataset is made publicly available to encourage further research.
Key Insights and Ideas
- Human Body Capacitance (HBC) as a valuable sensor: The integration of HBC sensing alongside IMUs proves effective for capturing nuanced body movements, especially for exercises that might be less apparent from wrist-worn IMUs alone.
- TinyML for complex tasks: Demonstrating that sophisticated deep learning models, like ResNets, can be effectively compressed and deployed on extremely low-power microcontrollers for complex tasks like multi-class activity recognition.
- The power of quantization: 8-bit quantization allows for a significant reduction in model size and computational cost with only a minor drop in accuracy, making edge deployment feasible.
- GAP8's potential for edge AI: The RISC-V based GAP8 platform shows a remarkable advantage in terms of inference speed and energy efficiency compared to traditional ARM Cortex-M microcontrollers for this specific task.
Key Results
- Accuracy: The model achieves an accuracy of up to 90.4% with full precision inference and 88.1% on the GAP8 platform with 8-bit quantization, showing a minimal drop.
- Inference Speed: On the GAP8 platform, inference takes only 3.2 ms at maximum clock speed, which is 18.9x faster than Cortex-M4 and 6.5x faster than Cortex-M7.
- Energy Efficiency: The energy consumed per inference on GAP8 is 0.41 mJ, significantly lower than 5.17 mJ on Cortex-M4 and 8.07 mJ on Cortex-M7.
- Throughput: The GAP8 platform achieves a throughput of 953.70 MMAC/s, a substantial improvement over the Cortex-M series.
Limitations and Future Work
- Dataset Imbalance: The dataset contains a significantly higher number of "Null" (non-exercise) instances than actual workout instances, which required specific weighting strategies during training.
- Sensor Placement: While the study collected data from wrist, leg, and pocket, the evaluation focused solely on wrist-worn data. Future work could explore the benefits of multi-sensor fusion from different body locations.
- Limited Workout Variety: Although eleven workouts were included, the authors suggest expanding the dataset to include a wider range of exercises and potentially more complex movements.
- Generalization: The study used a leave-one-user-out cross-validation strategy, which is a good step towards generalization, but further validation on larger and more diverse user groups would be beneficial.
- Real-world deployment: While promising, the results are based on controlled experimental setups. Real-world deployment might introduce additional challenges like varying environmental conditions and user motion patterns.
Summary updated