Human Activity Recognition Based on Motion Sensor Using U-Net

Yong Zhang, Zhao Zhang, Yu Zhang, Jie Bao, Yifan Zhang, Haiqin DengIEEE Access 2019 · OpenReview

Human Activity Recognition (HAR) is a cornerstone of modern wearable technology, powering everything from fitness tracking to clinical health monitoring. However, traditional methods often struggle with precision, particularly during activity transitions. A recent study by Yong Zhang and colleagues introduces a paradigm shift: treating sensor data as an image and applying the U-Net architecture to achieve sample-by-sample, or "dense," prediction.

The Multi-Class Window Problem

Most HAR systems utilize a sliding window approach. This involves taking a fixed-duration snapshot of sensor data (e.g., 3 seconds) and assigning it a single label. While simple, this method suffers from the "multi-class window problem." When a window contains more than one activity—such as a person stopping a walk to stand still—the system must choose one label, often misrepresenting the transition or completely missing short-duration activities.

Comparison of sliding window labeling versus dense labeling.

As shown in the figure above, sliding window labeling often relies on the "most appearing class" or "last appearing class," which leads to inaccuracies at boundaries. In contrast, dense labeling assigns a specific class to every individual sampling point, preserving the temporal integrity of the data.

A U-Net Framework for Dense Prediction

To implement dense prediction, the researchers adapted the U-Net architecture, originally designed for medical image segmentation. The framework involves three key stages:

  1. Data Mapping: Multi-channel sensor data (e.g., tri-axial acceleration) is mapped into a single-pixel column "image." This allows 1D time-series data to be processed by 2D convolutional layers.
  2. Encoder-Decoder with Skip Connections: The U-Net's unique structure is critical. The encoder captures high-level features while reducing resolution, and the decoder reconstructs the data to its original resolution. Crucially, skip connections pass high-resolution information from the encoder directly to the decoder, ensuring that the exact timing of activity changes is preserved.
  3. Post-Correction: To mitigate "jitter" or fragmentation in the sample-level labels, a post-correction algorithm analyzes boundary correlations to smooth the final output.
The U-Net architecture adapted for dense prediction of human activity from sensor data.

Experimental Results and Datasets

The model was rigorously tested against four datasets:

  • WISDM & UCI HAPT: Standard benchmarks for daily activities and complex transitions.
  • OPP Gesture: A dataset focused on short, discrete movements.
  • Sanitation: A new dataset introduced by the authors, capturing the diverse work activities of sanitation workers (e.g., sweeping, dumping trash).

The U-Net approach consistently outperformed traditional machine learning (SVM, Random Forest) and other deep learning models (CNN, LSTM, FCN). On the OPP Gesture dataset, U-Net achieved an accuracy of 94.7%, significantly higher than the ~80-90% typical of sliding-window methods. It also showed remarkable robustness in identifying "minority classes"—rare activities that are often ignored by traditional models.

Conclusion

By moving beyond the limitations of sliding windows, the U-Net framework provides a high-precision solution for HAR. Its ability to perform dense, sample-level prediction makes it particularly effective for capturing short-term activities and precise transitions, paving the way for more responsive and accurate wearable applications.

Summary updated

Abstract

Traditional human activity recognition (HAR) based on a motion sensor adopts sliding window labeling and prediction. This method faces the multi-class window problem, which mistakenly labels different classes of sampling points within a window as a class. In this paper, we propose a novel HAR method based on U-Net to overcome the multi-class problem, performing activity labeling and prediction of each sampling point. The motion sensor data collected from the wearable sensors are mapped into an image with the single-pixel column and multi-channel, and then, it is input into the U-Net network to complete the pixel-level activity recognition function. We design a complete HAR framework based on U-Net to realize the dense prediction of motion sensor data, including data preprocessing, dense prediction, and post analysis. In order to further improve the dense prediction performance, we propose the post-correction algorithm for the dense prediction results on the basis of the activity misalignment analysis. The extensive experimental results demonstrate that our U-Net method performs better than the traditional machine learning and deep learning methods based on the sliding window prediction. And it also outperforms full convolutional network (FCN), SegNet, and Mask R-CNN based on the dense prediction on the four datasets. Moreover, it also shows the better robustness and excellent performance of recognition on the short-term activities and minority classes. We release a new dataset named Sanitation, which includes seven types of daily work activity data of sanitation workers to evaluate the HAR algorithm’s performance.
BibTeX
@inproceedings{zhang2019human,
  author = {Yong Zhang and Zhao Zhang and Yu Zhang and Jie Bao and Yifan Zhang and Haiqin Deng},
  title = {{Human Activity Recognition Based on Motion Sensor Using U-Net}},
  booktitle = {IEEE Access},
  year = {2019},
  url = {https://openreview.net/forum?id=jexNU7ZGPo},
}
Figure