Abstract
Memorability measures how easily an image is retained in memory after a brief glance. While previous research has identified visual features that make objects or faces memorable, predicting the memorability of outdoor natural scenes remains a challenge due to the lack of distinct focal points. This paper introduces the Large-scale Natural Scene Image Memorability (LNSIM) database and DeepNSM, an end-to-end deep neural network that leverages scene category information to significantly improve memorability prediction for natural landscapes.
The Challenge of Natural Scene Memorability
Most image memorability research focuses on "object-centric" images where specific subjects—like people or animals—act as memory anchors. Outdoor natural scenes, such as deserts or mountain ranges, lack these salient anchors, making them harder for traditional models to evaluate. Existing state-of-the-art models like MemNet often struggle with these "background-only" images because they cannot easily identify which regions are worth remembering.
The LNSIM Dataset
To address this gap, the authors established the LNSIM database, the largest of its kind. It contains 2,632 images across 71 categories. Ground-truth memorability scores were collected using a "memory game" protocol where at least 80 participants per image identified repeated visuals in a stream of images.
DeepNSM: A Category-Aware Architecture
The core innovation of this research is the DeepNSM (Deep Natural Scene Memorability) predictor. Through feature analysis, the authors discovered that high-level scene categories are the strongest predictors of memorability in natural landscapes. DeepNSM utilizes a dual-branch architecture:
- Baseline Branch: A fine-tuned MemNet that extracts general spatial and visual features.
- Category Branch: A ResNet-based module fine-tuned on the "Places" dataset to identify specific scene types (e.g., glacier, marsh).
By fusing these generic deep features with category-specific data, the model can better understand the "macroscopic" context that the human brain uses to encode memories of environments.
Key Insights: The Role of Scene Category
A striking finding of the study is the correlation between memorability and rarity. Scenes that are rarely encountered in daily life or described less frequently in literature—such as an aurora or a unique rock arch—tend to be significantly more memorable than common landscapes like forests or mountains.
Furthermore, the research suggests that standard visual saliency (what catches the eye) does not equate to memorability in natural scenes. A visually striking element might attract immediate attention without making the overall scene stick in the viewer's mind.
Performance and Results
DeepNSM significantly outperforms previous methods. On the LNSIM dataset, it achieved a Spearman Rank Correlation Coefficient (SRCC) of 0.58, compared to 0.43 for MemNet and 0.15–0.33 for traditional handcrafted features. The model also demonstrated strong generalization, maintaining an SRCC of 0.55 when tested on independent datasets. With human consistency on this task measured at 0.78, DeepNSM represents a major step toward human-level performance in predicting subjective image memorability.
Conclusion
By shifting the focus from object-level saliency to macroscopic scene categories, this research provides a new framework for understanding how we remember the natural world. While challenges remain—such as handling low-contrast images or repeating patterns—the LNSIM dataset and DeepNSM model provide essential tools for fields ranging from tourism marketing to automated image curation.
Summary updated