Modifying the Memorability of Face Photographs

Aditya Khosla, Wilma A. Bainbridge, Antonio Torralba, Aude OlivaICCV 2013 · OpenReview

Abstract

Contemporary life bombards us with many new images of faces every day, which poses non-trivial constraints on human memory. The vast majority of face photographs are intended to be remembered, either because of personal relevance, commercial interests or because the pictures were deliberately designed to be memorable. Can we make a portrait more memorable or more forgettable automatically? Here, we provide a method to modify the memorability of individual face photographs, while keeping the identity and other facial traits (e.g. age, attractiveness, and emotional magnitude) of the individual fixed. We show that face photographs manipulated to be more memorable (or more forgettable) are indeed more often remembered (or forgotten) in a crowd-sourcing experiment with an accuracy of 74%. Quantifying and modifying the 'memorability' of a face lends itself to many useful applications in computer vision and graphics, such as mnemonic aids for learning, photo editing applications for social networks and tools for designing memorable advertisements.

Introduction

In an era of visual saturation, some faces linger in our memory while others vanish instantly. While common intuition suggests that memorability is tied to subjective beauty or distinctiveness, recent research indicates it is an intrinsic property of an image, remarkably consistent across different viewers. This paper addresses a novel challenge: can we algorithmically control this property? Specifically, the goal is to take a portrait and adjust its memorability without altering the subject's identity or key social attributes like age, gender, and attractiveness.

Methodology

The researchers developed a data-driven optimization approach to modify face photographs. The system relies on three core pillars:

  1. Face Representation: Using Active Appearance Models (AAMs), the system decomposes a face into its "shape" (landmark coordinates) and "appearance" (texture and pixel data). This allows for the synthesis of realistic new faces by navigating a low-dimensional mathematical space.
  2. A Refined Memorability Metric: The authors identified that some faces are "familiar"—causing high false alarms—rather than truly "memorable." They introduced a metric that subtracts false alarms from the hit rate, significantly improving the accuracy of their memorability predictors.
  3. Multi-Objective Optimization: To modify a face, the algorithm minimizes a cost function that balances three competing goals: identity preservation, target memorability, and attribute consistency (keeping traits like age and emotional expression constant).

Given the non-linear relationship between pixels and memory, the team employed a sampling-based optimization. The algorithm probes the mathematical space around the original face, iteratively refining candidates to find the optimal "memory face lift."

Key Results

The effectiveness of the method was validated through large-scale human experiments using the 10k US Adult Faces Database. In a crowd-sourcing test, images modified to be "more memorable" were correctly identified as such by participants in 74% of cases.

Comparison of original face photographs with their less and more memorable counterparts.

The results highlight several critical insights:

  • Subtlety is Key: As seen in the figure above, the modifications are often too subtle for a human to describe verbally, yet they have a measurable impact on memory.
  • Beyond Distinctiveness: While distinctiveness is a factor, it is not the sole driver of memorability. Global features like Histograms of Oriented Gradients (HOG) and SIFT are essential for capturing the complex textures that make a face stick.
  • Independent Control: The system successfully decoupled memorability from other traits. A face can be made more memorable without inadvertently making it look older or less attractive.

Conclusion and Future Work

This work demonstrates the first automated method for systematically manipulating the memorability of face photographs. While the computational cost of extracting dense global features remains high, the potential applications are vast—from designing more effective advertisements to creating mnemonic aids for education. Future research may extend these techniques to full-body photography or video, further exploring the holistic and elusive nature of human memory.

Summary updated

Abstract

Contemporary life bombards us with many new images of faces every day, which poses non-trivial constraints on human memory. The vast majority of face photographs are intended to be remembered, either because of personal relevance, commercial interests or because the pictures were deliberately designed to be memorable. Can we make a portrait more memorable or more forgettable automatically? Here, we provide a method to modify the memorability of individual face photographs, while keeping the identity and other facial traits (e.g. age, attractiveness, and emotional magnitude) of the individual fixed. We show that face photographs manipulated to be more memorable (or more forgettable) are indeed more often remembered (or forgotten) in a crowd-sourcing experiment with an accuracy of 74%. Quantifying and modifying the 'memorability' of a face lends itself to many useful applications in computer vision and graphics, such as mnemonic aids for learning, photo editing applications for social networks and tools for designing memorable advertisements.
BibTeX
@inproceedings{khosla2013modifying,
  author = {Aditya Khosla and Wilma A. Bainbridge and Antonio Torralba and Aude Oliva},
  title = {{Modifying the Memorability of Face Photographs}},
  booktitle = {IEEE/CVF International Conference on Computer Vision},
  year = {2013},
  url = {https://openreview.net/forum?id=B1Nqe-MdWB},
}
Figure