In deep learning, we typically think of the "forward pass" as a one-way street: an input image enters the network, passes through successive layers of artificial neurons, and produces a final prediction. To understand what an individual neuron computes, the traditional approach has been to feed a massive dataset through the network and observe which natural images cause it to fire most intensely.
However, this approach is fundamentally limited by the diversity of the dataset. What if the true, ideal trigger for a neuron is an abstract pattern or a specific combination of features that does not exist in nature?
To answer this question, researchers invert the forward pass. Instead of treating the input image as fixed and the network weights as learnable parameters, they fix the weights and treat the input image as the variable to be optimized. By defining an objective function that targets the activation of a specific neuron, voxel, or brain region, they can use gradient ascent to sculpt an image from scratch. This core idea—known as activation maximization or gradient-based feature synthesis—allows us to bypass natural constraints, revealing the internal representations of artificial networks and providing a powerful tool to causally control biological brains.
The Mechanics of Inverting a Network
The reason activation maximization works is that deep neural networks are fully differentiable computational graphs. Just as we can compute the gradient of a loss function with respect to the weights to train a model, we can compute the gradient of a specific neuron's activation $A(x)$ with respect to the input pixels $x$.
$$\Delta x \propto \nabla_x A(x)$$
By iteratively adding this gradient to a starting image (often initialized as random white noise), the input gradually transforms into a pattern that maximizes the target neuron's response.
This technique has proven invaluable for visualizing what complex models learn. For example, Christoph Feichtenhofer et al. applied activation maximization to video models in What Have We Learned From Deep Representations for Action Recognition?, synthesizing preferred inputs for spatiotemporal features to reveal how networks fuse motion and appearance. Similarly, the authors of Rotation-invariant clustering of neuronal responses in primary visual cortex used optimized inputs to visualize the non-linear computational motifs learned by neural clusters, providing a clear visual interpretation of abstract network representations.
The Challenge of Unconstrained Optimization
While mathematically straightforward, optimizing raw pixels directly often yields a surprising result: high-frequency, surrealistic noise patterns that look like television static to humans but drive the target neuron to maximum activation.
This phenomenon occurs because artificial neural networks learn idiosyncratic invariances that diverge sharply from human perception. As Jenelle Feather et al. demonstrated in Model metamers illuminate divergences between biological and artificial neural networks, it is possible to synthesize "model metamers"—stimuli that produce identical internal representations within a network but are perceived by humans as completely meaningless noise.
To force the optimization process to generate semantically meaningful, interpretable images, researchers introduce a generative prior. Instead of optimizing individual pixels in an unconstrained space, the optimization occurs within the low-dimensional latent space of a pre-trained generative model, such as a Generative Adversarial Network (GAN) or a Latent Diffusion model. The gradient of the target neuron's activation is backpropagated through the neural network and into the generative model's latent space, steering the generator to synthesize realistic images that satisfy the activation objective.
Controlling the Biological Brain via Digital Twins
One of the most profound realizations in computational neuroscience is that activation maximization is not restricted to artificial networks; it can be used to control the biological brain. This is achieved by constructing an image-computable "digital twin" of a sensory system.
First, a deep neural network is trained as an encoding model to predict how biological neurons or brain regions respond to visual stimuli. Because this encoding model is differentiable, it can serve as a proxy for the living brain. Researchers can perform activation maximization in silico on the model to synthesize entirely novel stimuli designed to drive biological neural activity into extreme states.
[Latent Prior / Noise] ──> [Generative Model] ──> [Synthesized Image]
│
▼
[Biological Brain] <── [Maximize Response] <── [Differentiable Encoding Model]
In a landmark study, Kohitij Kar et al. (Neural Population Control via Deep Image Synthesis) used this exact method to synthesize "alien" super-stimuli for primate visual neurons. When presented to the animals, these synthesized images drove target neurons to firing rates averaging 39% higher than any natural image could achieve. As reviewed by Kohitij Kar et al. in The Quest for an Integrated Set of Neural Mechanisms Underlying Object Recognition in Primates, these task-optimized "SMART" models act as mechanistic hypotheses that allow scientists to probe the functional limits of biological vision.
This closed-loop synthesis loop has been successfully deployed across various species and imaging modalities:
- Cortical Regions: Matthew W. Shinkle et al. (Visualizing and Controlling Cortical Responses Using Voxel-Weighted Activation Maximization) adapted this technique to human fMRI, treating the regression weights of a voxel-encoding model as a final network layer to synthesize optimal visual inputs that successfully drive targeted brain regions in live subjects.
- Cellular Selectivity: Gabriel Kreiman et al. developed XDream (XDream: Finding preferred stimuli for visual neurons using generative networks and gradient-free optimization), a gradient-free alternative that pairs a generative network with a genetic algorithm to evolve super-stimuli in a closed loop, demonstrating that synthesized images often represent exaggerated, distilled versions of a neuron's preferred features.
- Multimodal Behaviors: In mouse models, Aiwen Xu et al. (Multimodal Deep Learning Model Unveils Behavioral Dynamics of V1 Activity in Freely Moving Mice) utilized gradient ascent to identify the joint visual and behavioral features (such as head orientation and running speed) that maximize primary visual cortex (V1) activity.
As highlighted by Alessia Celeghin et al. in Convolutional neural networks for vision neuroscience: significance, developments, and outstanding issues, these closed-loop experiments establish direct causal links between visual features and neural responses, validating the role of deep networks as accurate models of the visual cortex. Early foundations for this paradigm were laid by Leon A. Gatys et al. (Texture Synthesis Using Convolutional Neural Networks), who demonstrated that inverting deep neural representations provides a powerful generative framework for investigating biological vision.
Scaling to Complex Semantics with Diffusion Models
When applied to large-scale human brain imaging, activation maximization helps map out complex, compositional semantics across the entire cortex. By guiding modern latent diffusion models with the gradients of neural encoding models, researchers can "hallucinate" what specific brain regions want to see without the bias of hand-picked stimulus sets.
For instance, Andrew F. Luo et al. introduced BrainDiVE (Brain Diffusion for Visual Exploration: Cortical Discovery using Large Scale Generative Models), a framework that injects neural encoding gradients directly into the iterative denoising process of a diffusion model. Similarly, Ethan Hwang et al. (In Silico Mapping of Visual Categorical Selectivity Across the Whole Brain) used diffusion-based synthesis to discover brain regions selective for highly nuanced, compositional concepts—such as "hands using tools"—moving far beyond traditional categories like "faces" or "places."
These synthesized super-stimuli often trigger responses that exceed natural baseline limits. Keith W Jamison et al. (Human brain responses are modulated when exposed to optimized natural images or synthetically generated images) confirmed that optimizing the latent space of a generative model creates super-stimuli that evoke significantly stronger high-level visual responses than any image found in natural datasets. Yukiyasu Kamitani (脳をくすぐるアート) argues that these machine-optimized images function as neural super-stimuli by bypassing natural perceptual constraints, directly manipulating neural response magnitudes to externalize brain-internal representations.
This generative inversion paradigm also enables high-throughput testing of domain-specific brain regions. Apurva Ratan Murty et al. (Computational models of category-selective brain regions enable high-throughput tests of selectivity) paired GANs with encoding models to evaluate category selectivity across millions of synthetic images. Conversely, the same underlying mapping can be run backward to decode subjective experiences: Furkan Özçelik et al. (Natural scene reconstruction from fMRI signals using generative latent diffusion) reversed the mapping between brain activity and generative latent spaces to reconstruct perceived natural scenes, while Johannes Mehrer et al. (Model-Guided Microstimulation Steers Primate Visual Behavior) used generative models to visualize the subjective hallucinations induced by direct cortical microstimulation.
Auditing and Red Teaming Artificial Networks
The exact same mechanics used to discover what the biological brain wants to see can be deployed to audit artificial networks for hidden bugs, biases, or security vulnerabilities.
In AI safety and interpretability, optimization-based feature synthesis acts as an independent auditor. Tong Bu (Red Teaming Deep Neural Networks with Feature Synthesis Tools) demonstrated that synthesizing inputs from scratch can expose out-of-distribution vulnerabilities and hidden malicious trojans that traditional dataset-dependent saliency methods completely miss.
Understanding these triggers is critical because networks can be intentionally hijacked using the same principles. Adnan Siraj Rakin et al. (TBT: Targeted Neural Network Attack With Bit Trojan) showed that an attacker can mathematically design a small input patch via targeted activation trigger optimization to maximize the firing rate of specific vulnerable neurons, forcing a compromised model to misclassify any input whenever that specific trigger patch is present.
[Candidate Text Description] ──> [Text-to-Image Model] ──> [Synthetic Stimulus]
│
▼
[Causal Verification] <─── [Measure Activation] <─── [Target Vision Network]
Finally, feature synthesis provides a rigorous way to validate automated explanations of neural behavior. In the Describe-and-Dissect pipeline, Nicholas Bai et al. (Interpreting Neurons in Deep Vision Networks with Language Models) used large language models to generate text descriptions of what a vision neuron detects, and then used a text-to-image model to synthesize visual stimuli based on those descriptions. By feeding these synthetic images back into the network, they could causally verify whether the description accurately captured the neuron's true trigger, ensuring that the explanation reflected a genuine causal mechanism rather than a coincidental correlation.
Key Foundational Work
Neural Population Control via Deep Image Synthesis (2018) — contributed 2 concept elements to this direction.
Texture Synthesis Using Convolutional Neural Networks (2015) — contributed 1 concept elements to this direction.
What Have We Learned From Deep Representations for Action Recognition? (2018) — contributed 1 concept elements to this direction.