Aiding autobiographic memory by using wearable devices

Jingyi Wang, Jiro TanakaarXiv 2017 · arXiv

Abstract

Lifelogging captures vast amounts of daily data, but retrieving meaningful memories remains a challenge. This paper proposes a method to filter lifelogging footage using heart rate spikes and GPS heat maps. By identifying "Special Moments" through physiological arousal and "Spatial Frequency" through location density, the system helps users rediscover forgotten events. Preliminary user studies show significant improvements in memory recall and user experience compared to standard chronological review.

Introduction

Lifelogging—the practice of capturing one's life through wearable cameras—promises a "perfect" digital memory. However, the reality is often a digital junk drawer: thousands of blurry, mundane photos that are tedious to review. The primary challenge is retrieval: how to find "special moments" in a sea of 8-hour-long recordings. Current systems often rely on simple chronological scrolling, which fails to account for the emotional or situational importance of an event. This research aims to bridge this gap by using physiological signals (heart rate) and location density (GPS heat maps) as filters to surface meaningful autobiographical memories.

Methodology

The researchers proposed a multi-stage pipeline to process lifelog data collected from two primary devices: a wearable camera and a heart rate monitor.

The hardware setup: a wearable Autographer camera and a Polar A360 heart rate monitor.

Data Processing Pipeline

  1. Robust Motion Deblurring: To mitigate "camera swing" inherent in wearable devices, a specific deblurring algorithm was applied to improve image quality before user review.
  2. The Special Moment Approach: This technique uses heart rate as a proxy for emotional or physical arousal. Heart rate data is normalized and averaged into 30-second windows to match the camera's capture frequency. Sudden spikes in heart rate are flagged as "Special Moments."
  3. The Spatial Frequency Approach: Instead of simple path tracking, the system generates heat maps to visualize "frequency awareness." This highlights locations where a user spent the most time, which often correlates with significant events like meetings or social gatherings.
Mapping heart rate spikes to specific lifelog images to identify 'Special Moments'.

Experimental Setup

The study utilized a custom dataset collected from four university students (3 male, 1 female). Each participant wore the devices for two days, approximately 8 hours per day, capturing lifelog images, GPS coordinates, and heart rate (BPM).

Results and Discussion

The approach was evaluated using a Likert scale (1-5) comparing it to a "Regular Review" (standard chronological viewing).

  • User Experience: Both the Special Moment and Spatial Frequency approaches scored significantly higher (~4.0/5.0) than the Regular Review (~3.0/5.0) in terms of meaningfulness, clarity, and intuitiveness.
  • Memory Recall: While the Regular Review provided a baseline for event recall, the Special Moment approach led to a significant increase in the discovery of "new" memories—events participants had forgotten until prompted by the biophysical cue.
  • Qualitative Feedback: Users found the heat map particularly intuitive for "location awareness," helping them reconstruct the narrative of their day more effectively than a simple photo list.

Conclusion

This research demonstrates that biophysical and spatial data can serve as powerful filters for overwhelming lifelog datasets. By leveraging the body's physiological responses and spatial habits, we can transform a "digital junk drawer" into a functional aid for autobiographical memory. Future work will focus on automating the extraction of special moments and incorporating additional signals like skin conductance to further refine detection.

Summary updated

BibTeX
@misc{wang2017aiding,
  author = {Jingyi Wang and Jiro Tanaka},
  title = {{Aiding autobiographic memory by using wearable devices}},
  year = {2017},
  eprint = {1706.01248},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1706.01248},
}
Figure