Memento: Augmenting Personalized Memory via Practical Multimodal Wearable Sensing in Visual Search and Wayfinding Navigation
Lists
Only you see this
Abstract
Working memory involves the temporary retention of information over short periods. It is a critical cognitive function that enables humans to perform various online processing tasks, such as dialing a phone number, recalling misplaced items' locations, or navigating through a store. However, inherent limitations in an individual's capacity to retain information often result in forgetting important details during such tasks. Although previous research has successfully utilized wearable and assistive technologies to enhance long-term memory functions (e.g., episodic memory), their application to supporting short-term recall in daily activities remains underexplored. To address this gap, we present Memento, a framework that uses multimodal wearable sensor data to detect significant changes in cognitive state and provide intelligent in situ cues to enhance recall. Through two user studies involving 15 and 25 participants in visual search navigation tasks, we demonstrate that participants receiving visual cues from Memento achieved significantly better route recall, improving approximately 20-23% compared to free recall. Furthermore, Memento reduced cognitive load and review time by 46% while also substantially reducing computation time (3.86 seconds vs. 15.35 seconds), offering an average of 75% effectiveness compared to computer vision-based cue selection approaches.
BibTeX
@inproceedings{ghosh2025memento,
author = {Indrajeet Ghosh and Kasthuri Jayarajah and Nicholas Waytowich and Nirmalya Roy},
title = {{Memento: Augmenting Personalized Memory via Practical Multimodal Wearable Sensing in Visual Search and Wayfinding Navigation}},
booktitle = {UMAP},
year = {2025},
doi = {10.1145/3699682.3728340},
}| Aiding autobiographic memory by using wearable devices | arXiv · 2017 |
| System Development for Capture and Retrieval of Life Log | The Journal of The Institute of Image Information and Television Engineers · 2007 |
| Encode-Store-Retrieve: Enhancing Memory Augmentation through Language-Encoded Egocentric Perception | arXiv · 2023 |
| EgoTrigger: Toward Audio-Driven Image Capture for Human Memory Enhancement in All-Day Energy-Efficient Smart Glasses | arXiv · 2025 |
| Memoro: Using Large Language Models to Realize a Concise Interface for Real-Time Memory Augmentation | arXiv · 2024 |
| LightMem-Ego: Your AI Memory for Everyday Life | arXiv · 2026 |