Introduction
Maintaining a healthy lifestyle is essential for preventing chronic diseases, yet tracking daily habits like diet, exercise, and hydration is notoriously difficult. Traditional mobile apps require tedious manual input, leading to high abandonment rates. Visual lifelogging—using wearable cameras to passively record daily activities—offers a promising alternative. However, existing datasets often capture images too infrequently (e.g., every 30 seconds), missing brief but vital health-related events.
Researchers have addressed these gaps by developing the Visual Lifelogging Dataset for Lifestyle Analysis (VLDLA) and a computer vision framework designed to automatically assess a person's daily routine. By modeling internal states such as hunger, thirst, and fatigue, the system can distinguish between balanced lifestyles and unhealthy, sedentary ones.
The VLDLA Dataset
The authors introduced VLDLA to provide a more continuous and dense record of daily life compared to previous egocentric datasets. It consists of 84,000 high-resolution images captured every 3 seconds from 8:00 am to 6:00 pm over seven consecutive days. The dataset includes 12 activity categories, such as using a computer, attending class, exercising, and drinking. To protect privacy, faces and screens are automatically blurred.
Methodology: Activity Recognition and Latent Fluents
The proposed system utilizes a two-stage pipeline for lifestyle analysis:
- Activity Recognition: The framework extracts scene features using ResNet50 and object features via Mask R-CNN. These are fused and processed by a BiLSTM-CRF (Bidirectional Long Short-Term Memory with Conditional Random Fields). The BiLSTM captures temporal correlations, while the CRF ensures logical consistency—for example, preventing the model from predicting two full meals within a few minutes.
- Lifestyle Analysis via Latent Fluents: The core innovation is the modeling of "latent fluents": hunger, thirst, and fatigue. These are mathematical functions that increase over time and decrease when the system detects a relevant activity (eating, drinking, or resting). A final "lifestyle score" is derived from the average of these states; lower cumulative "need" states result in a higher health score.
Key Insights and Results
The study demonstrates that a 30-second capture interval is insufficient for health modeling, as it frequently misses "micro-activities" like taking a sip of water. By increasing the density to 3 seconds, the model can more accurately track hydration and other brief habits. Furthermore, combining scene context with object detection significantly improves the ability to distinguish between similar sedentary tasks, such as reading versus using a smartphone.
The activity recognition model achieved an overall accuracy of 85.57%. It was particularly effective at identifying "Resting" (0.97 F1-score) and "Exercising" (0.96 F1-score). However, it faced challenges with "Drinking" (0.38 F1-score) and "Social" (0.46 F1-score) due to the small size of objects like cups and the visual complexity of social interactions.
Limitations and Future Directions
Currently, the dataset is limited to daytime hours (8:00 am to 6:00 pm), missing potentially unhealthy late-night habits due to privacy concerns in residential settings. Additionally, detecting small objects remains a technical hurdle. Future work aims to integrate multimodal data from wearable sensors (e.g., heart rate) to ground the "latent fluents" in actual physiological measurements rather than time-based heuristics alone.
Summary updated