Introduction
Human Activity Recognition (HAR) has achieved significant success in controlled laboratory settings. However, transitioning these models to "free-living" environments—the real world—presents a major challenge: the "noisy label" problem. When participants self-report activities, they use inconsistent, open-ended text. For example, "walking," "strolling," and "going to the shop" might all describe the same physical state but appear as distinct labels in a dataset. This results in a sparse, high-cardinality vocabulary that standard machine learning models struggle to generalize, especially under the constraints of low-frequency sensor sampling common in wearable devices.
While Large Language Models (LLMs) like GPT-4 are often employed to clean and standardize text, they frequently lack the domain-specific physiological understanding required for HAR. To address this, we present FuSE-MET (Fusion of Semantic Embeddings and MET Values), a framework that cleans noisy labels by integrating linguistic meaning with physical intensity data.
The FuSE-MET Framework
The core innovation of FuSE-MET is the mapping of user-provided labels into a multi-dimensional feature vector that captures both semantic and physiological characteristics.
The framework utilizes a weighted fusion formula to represent each activity $a_i$: $f(a_i) = (1 - \lambda) \cdot w(a_i) + \lambda \cdot m(a_i)$
Here, $w(a_i)$ represents BERT-based word embeddings that capture linguistic relationships (e.g., the proximity of "shopping" to "buying"). The term $m(a_i)$ represents Metabolic Equivalent of Task (MET) values, a clinical standard for measuring energy expenditure. The hyperparameter $\lambda$ balances these two inputs. Following fusion, labels are grouped via K-means clustering to create a refined, manageable label space for training classifiers like XGBoost or SVM on smartphone accelerometer and gyroscope data.
Experimental Evaluation
The framework was evaluated using data from a clinical study of patients with cardiovascular disease. Sensor data was segmented into 5-second windows with a 50% duty cycle to simulate battery-saving constraints. The original dataset contained over 36 unique, noisy labels provided by users through active learning.
Key Results
FuSE-MET consistently outperformed both the "no-fusion" baseline and GPT-4-based label merging across various cluster sizes ($K$).
As shown in the results, the best performance was achieved at $K=3$ using an XGBoost classifier, yielding an F1-score of 0.77—significantly higher than the 0.51 achieved by a GPT-4-based baseline. At a coarser granularity ($K=2$), the model achieved a near-perfect accuracy of 0.98 using an SVM. In contrast, attempting to classify the original 36 noisy labels without any fusion resulted in a dismal F1-score of 0.18, underscoring the necessity of label cleaning in free-living HAR.
Discussion and Insights
The superiority of FuSE-MET over LLMs suggests that domain knowledge acts as a vital regularizer. While GPT-4 is adept at language, it lacks the "physical intuition" to distinguish between activities that are semantically similar but physically distinct. Integrating MET values prevents the model from incorrectly merging low-intensity and high-intensity activities.
Furthermore, the optimal $\lambda$ values (typically between 0.2 and 0.5) indicate that while physical intensity is a critical anchor, semantic meaning remains the primary driver for successful activity classification. By reducing label complexity, FuSE-MET allows classical classifiers to achieve high performance without overfitting to the noise inherent in raw user input.
Conclusion and Future Work
FuSE-MET demonstrates that domain-informed label fusion is a powerful tool for real-world HAR. Future research will focus on optimizing the mapping of MET values into the embedding space, testing the framework on more diverse populations, and developing methods to automatically learn the optimal balance between semantic and physical features.
Summary updated