Why do some images stick in our minds forever, while others fade away almost instantly? For a long time, visual memorability was thought to be entirely subjective—dependent on an individual's personal history, emotional state, or the cultural context of the image. However, computational research has revealed that memorability is largely an intrinsic image property. It can be quantified, predicted, and systematically manipulated.
The core idea behind post-processing memorability enhancement is that we can alter the visual features of an image to increase its likelihood of being remembered by a human observer, all while preserving its original semantic meaning and realism.
The Information-Preservation Trade-Off
To understand why certain visual modifications enhance or degrade memorability, we have to look at how the human visual system encodes information. The brain relies heavily on high-frequency details, sharp edges, and distinct color information to differentiate and retain visual representations.
As Oleksii Sidorov demonstrates, standard photo editing techniques often inadvertently degrade an image's mnemonic impact. Operations that simplify an image or remove visual information—such as background blurring, grayscaling, or vignetting—consistently result in lower memorability scores. When fine-grained visual details are stripped away, the image loses the unique, distinguishing features necessary for long-term retention.
[Original Image] ───► (Sharpening / Edge Boost) ───► Higher Memorability (Preserves/Enhances Detail)
───► (Blurring / Grayscaling) ───► Lower Memorability (Destroys High-Frequency Info)
Conversely, sharpening stands out as a classical transformation that reliably boosts memorability. By enhancing edges and local contrast, sharpening amplifies the saliency of facial reliefs, shadows, and textures. This makes the visual components more distinct, directly aiding the cognitive processes responsible for memory encoding.
Mechanisms of Memorability Manipulation
How do we systematically optimize an image's memorability without breaking its realism or changing what the image actually depicts? Researchers have developed several distinct paradigms to solve this problem:
1. Feature Warping
For highly specific domains like portrait photography, memorability can be tuned by making subtle geometric adjustments to the subject matter itself. In Modifying the Memorability of Face Photographs, Aditya Khosla et al. introduced a data-driven optimization approach that subtly warps facial features. By navigating a multi-dimensional space of facial configurations, their system can increase or decrease a face's memorability while strictly preserving the individual's identity, age, and perceived attractiveness.
2. Compositional and Color Remixing
In broader digital media contexts, optimization often occurs through framing and color adjustments. By analyzing how famous photographs are remixed across the web, Lisa Saleh et al. in The Computational Memorability of Iconic Images showed that tighter cropping and color filtering significantly alter an image's intrinsic memorability. Tighter crops eliminate distracting background elements and force the viewer's focus onto the salient subject, while color filters enhance dramatic contrast, maximizing visual impact.
3. Predictive Style Transfer
Instead of relying on manual adjustments or rigid geometric warping, deep learning can be used to discover optimal stylistic transformations. In How to Make an Image More Memorable? A Deep Style Transfer Approach, Aliaksandr Siarohin et al. introduced a framework ("S-cube") that pairs style transfer networks with memorability predictors.
A key insight of their work is that the system should not just optimize for a high absolute memorability score; instead, it should predict the relative change or improvement ($\Delta$) in memorability that a specific style seed would cause. By targeting the maximum positive delta, the network selects stylistic modifications that fundamentally enhance the image's memorability while keeping the underlying content recognizable.
┌─────────────────┐ ┌──────────────────┐ ┌─────────────────┐
│ Input Image ├─────►│ Style Transfer ├─────►│ Optimized Image │
└────────┬────────┘ └────────▲─────────┘ └────────┬────────┘
│ │ │
▼ │ ▼
┌─────────────────┐ ┌────────┴─────────┐ ┌─────────────────┐
│ Memorability │ │ Select Style │ │ Memorability │
│ Baseline Score ├─────►│ Maximized Δ │◄─────┤ Predicted Score │
└─────────────────┘ └──────────────────┘ └─────────────────┘
Why Visual Optimization Works
Ultimately, these techniques succeed because they bridge the gap between computer vision and human cognitive psychology. As established by Aditya Khosla et al. in their work on Understanding and Predicting Image Memorability at a Large Scale, deep neural networks like MemNet can accurately map out the specific regions of an image that drive human recall.
Once these high-impact regions are identified, optimization algorithms can selectively highlight them, sharpen their boundaries, or apply dramatic stylistic textures to them. By mathematically maximizing the predictive scores of models like MemNet, these systems are effectively tuning the image to exploit the natural biases of the human memory system—ensuring that the visual information is prioritized during cognitive encoding.
Key Foundational Work
Understanding and Predicting Image Memorability at a Large Scale (2015) — contributed 2 concept elements to this direction.
Modifying the Memorability of Face Photographs (2013) — contributed 1 concept elements to this direction.