TL;DR
This paper proposes a novel method for training neural networks to learn "complex-like" receptive fields, similar to those found in the visual cortex, by optimizing for temporal smoothness in natural video sequences. The approach successfully extracts translation-invariant features, suggesting that temporal coherence in natural stimuli is a key driver for learning these complex visual processing units.
What Problem Does This Paper Address?
Understanding how the brain learns to process visual information, particularly how it achieves invariance to transformations like translation and contrast changes, is a fundamental challenge in computational neuroscience. While many models can learn to be selective for specific stimuli, learning representations that are robust to irrelevant variations remains difficult. This paper tackles the problem of learning translation invariance from natural video data, aiming to replicate the properties of complex cells in the visual cortex, which are known for their invariance to stimulus position and contrast.
Methods
The authors employ a network of "subspace energy detectors" trained using gradient ascent on an objective function that maximizes temporal smoothness. The stimuli are 10x10 patches extracted from natural videos recorded by a camera mounted on a cat's head. These patches are preprocessed using PCA and whitening. The core of the method lies in defining an objective function, $O_{time}$, which penalizes rapid changes in the output of the network's neurons over time. By optimizing this function, the network is encouraged to find features that change slowly across consecutive video frames. The neurons themselves are modeled as computing the 2-norm of their sub-unit activities, and the sub-units are forced to be orthonormal in whitened space.
Datasets
The primary dataset used is natural video footage recorded by a camera mounted on a cat's head. The paper mentions that these videos are sampled at 25 Hz and that patches are extracted from these sequences. No new benchmark datasets are introduced.
Key Insights and Ideas
- Temporal Smoothness as a Learning Principle: The paper strongly suggests that optimizing for temporal smoothness in natural video sequences is a powerful principle for learning complex visual features. This is a more biologically plausible mechanism than some prior approaches that relied on explicit invariance constraints.
- Emergence of Translation Invariance: The learned receptive fields exhibit significant translation invariance, meaning the network's response is largely independent of where the stimulus appears in the visual field. This is a hallmark of complex cells.
- Contrast Polarity Invariance (Built-in): The network inherently learns contrast polarity invariance due to the use of an even norm in the neuron's transfer function. This means the response to a bright bar is similar to a dark bar.
- Natural Stimuli are Crucial: Shuffling the video frames to destroy temporal coherence leads to a loss of the learned complex-like receptive fields, underscoring the importance of temporal correlations in natural videos for this learning process.
- Connection to Existing Models: The learned receptive fields are compared to those obtained by other methods like Independent Subspace Analysis (ISA) and the Adaptive-Subspace SOM (ASSOM), highlighting similarities and differences in their approaches and outcomes.
Key Results
- The trained network produces receptive fields for its sub-units that resemble simple cells.
- The neurons, formed by combining these sub-units, exhibit properties similar to complex cells, demonstrating translation invariance.
- The $\theta$-r diagrams (representing response strength as a function of orientation and position) for the complex cells show a broad tuning to position, indicating invariance.
- Increasing the temporal lag between paired frames ($\Delta t$) leads to decreased orientation specificity and increased position specificity, as expected when temporal correlations diminish.
- The choice of PCA dimension (from 10 to 30) did not significantly affect the position and orientation specificity of the learned cells.
Limitations and Future Work
- PCA Dimension Choice: While the choice of PCA dimension didn't significantly alter results, the paper notes that different dimensions might require different subspace sizes and numbers of neurons for optimal coverage.
- Dynamic Subspace Size: The authors suggest that incorporating a dynamic subspace size into the objective function could help recruit the optimal number of sub-units.
- Norm of Subspaces: The current work uses a 2-norm for the subspace transfer function. Future work could explore learning the norm itself, potentially leading to learned nonlinearities and explicit contrast polarity invariance.
- Max Function: The paper speculates that learning other nonlinearities, like the max function (infinity norm), might also be achievable with the same objective function.
- Biological Plausibility: The temporal smoothness objective is noted as being compatible with local information processing in the mammalian cortex.
Summary updated