Experience-driven formation of parts-based representations in a model of layered visual memory

Jenia Jitsev, Christoph von der Malsburg

Abstract

This paper introduces a self-organizing, two-layer neural model designed to learn hierarchical representations of complex objects, such as faces, through experience. By integrating bidirectional synaptic plasticity, homeostatic regulation, and oscillatory "winner-take-all" dynamics, the model autonomously develops a parts-based memory system. Results demonstrate that this recurrent architecture achieves faster training stability and superior generalization to novel facial expressions compared to traditional feed-forward models, offering insights into how the brain coordinates local synaptic changes to form coherent global identities.

Introduction

A fundamental challenge in cognitive neuroscience is understanding how local synaptic mechanisms coordinate to represent complex, multi-part objects. The human visual system employs a "parts-based" approach, recognizing individual features like eyes and noses to construct a global identity. However, modeling how these parts and their spatial relationships are learned simultaneously without external supervision—the "binding problem"—remains difficult. Furthermore, biological systems must maintain sparse representations to avoid memory interference. This study proposes a Layered Visual Memory Model that addresses these issues through self-organization and recurrent connectivity.

Model Architecture and Methods

The proposed model consists of two primary hierarchical levels inspired by the ventral visual stream. The lower layer, analogous to the Inferior Temporal (IT) cortex, contains "bunch columns" that process local facial landmarks using Gabor filters. These units represent "Local Features." The higher layer, representing the Prefrontal Cortex (PFC), contains an "identity column" that integrates signals from the lower layer to form a global representation of identity.

Figure 1: The hierarchical architecture of the model, illustrating the flow of information from V1-V2 through IT (local features) to PFC (identity), including bidirectional and lateral connections.

Key technical components include:

  • Bidirectional Plasticity: Synaptic updates follow rules inspired by BCM and ABS theories, allowing for both long-term potentiation (LTP) and depression (LTD).
  • Activity Homeostasis: To ensure efficient memory distribution, a homeostatic mechanism adjusts neuronal excitability, preventing individual units from dominating the representation.
  • Recurrent Connectivity: Unlike feed-forward networks, this model utilizes lateral and top-down connections to provide contextual support and stabilize learning.

Oscillatory Dynamics and the Decision Cycle

The model employs gamma-range oscillations to modulate competition between units. This creates a structured "decision cycle" where units compete to become the "winner" for a specific feature or identity.

Figure 2: The oscillatory decision cycle showing the temporal evolution of excitatory (Exct.) and inhibitory (Inh.) signals, defining the window for competitive selection.

During each cycle, the interplay between excitation and inhibition ensures that only the most relevant units are activated, facilitating sparse coding and reducing interference between stored memories.

Results and Insights

The model was evaluated using the AR Face Database. A key finding was the "maturation sequence": the system learns parts first, followed by lateral associations, and finally the global identity and top-down feedback. This structured learning allows the fully recurrent model to reach stable performance 33% faster than a purely feed-forward counterpart.

In terms of generalization, the recurrent model significantly outperformed feed-forward architectures on "duplicate" views (images taken weeks apart) and varying expressions. Specifically, the feed-forward model's error rate was 38% higher for smiling faces and 62% higher for sad faces. Successful recall was characterized by high correlation between bottom-up and top-down signals, whereas failures were marked by a breakdown in this coordination.

Conclusion

This research demonstrates that recurrent connections and oscillatory dynamics are crucial for the efficient, autonomous formation of hierarchical object representations. While the current model relies on pre-aligned landmarks, future work will focus on achieving transformation invariance (handling rotation and scaling) and extending the hierarchy to more biological layers, potentially incorporating active vision mechanisms like saccadic eye movements.

Summary updated

BibTeX
@article{jitsev2009experience,
  author = {Jenia Jitsev and Christoph von der Malsburg},
  title = {{Experience-driven formation of parts-based representations in a model of layered visual memory}},
  journal = {Frontiers Comput. Neurosci.},
  year = {2009},
  doi = {10.3389/neuro.10.015.2009},
}
Figure