Divisive Normalization Circuits Faithfully Represent Auditory and Visual Sensory Stimuli

Aurel A. Lazar, Tingkai Liu, Yiyin ZhoubioRxiv 2022 · DOI

TL;DR

This paper demonstrates that Divisive Normalization Processors (DNPs), a common model for sensory processing in the brain, are mathematically invertible. This means that the original sensory input (auditory or visual) can be perfectly reconstructed from the DNP's output, provided enough samples are available, suggesting that DNPs don't inherently lose information.

What Problem Does This Paper Address?

Divisive normalization is a fundamental neural computation found in various sensory systems, like vision and hearing, and is thought to be crucial for adaptation and efficient coding. However, the highly non-linear nature of this operation has made it difficult to quantitatively assess how much information is preserved. Specifically, researchers have questioned whether this transformation leads to information loss, especially in dynamic settings where the input signal changes over time. This paper aims to resolve this by investigating the invertibility of DNPs, essentially asking: can we get the original sensory signal back from the processed signal?

Methods

The authors leverage a general mathematical framework called the Divisive Normalization Processor (DNP) to model both auditory and visual sensory stimuli. They first establish that the DNP framework encompasses a broad range of neural circuits and phenomenological models, including the widely used Linear-Nonlinear (LN) cascade. The core of their methodology involves formulating the problem of recovering the original input stimulus from the DNP's output. This is approached by:

  1. Formalizing DNP Input/Output: They mathematically describe the input-output relationship of DNPs for both temporal (auditory) and spatio-temporal (visual) signals, considering feedforward and feedback architectures.
  2. Recovery Problem Formulation: They frame the recovery as finding the original input signal u given the DNP's output samples v(tk). This involves solving for u using known operators that define the DNP.
  3. Evaluation Criteria: The success of the recovery is quantitatively measured using Signal-to-Noise Ratio (SNR) and Peak Signal-to-Noise Ratio (PSNR).
  4. Algorithms for Recovery:
    • First-Order Kernels: For DNPs with first-order kernels, they employ a regularized regression approach. This involves solving a convex optimization problem, where sampling of the DNP output can be asynchronous.
  5. Second-Order Kernels: For DNPs with second-order kernels, they formulate the recovery as a rank-minimization problem, using a nuclear-norm convex relaxation. This approach leverages the fact that certain components of the DNP's internal structure are rank-1 matrices.

Datasets

Not discussed in the paper. The paper focuses on theoretical modeling and simulation with synthetic stimuli.

Key Insights and Ideas

  • DNPs are Invertible: The most significant finding is that DNPs, under certain conditions (sufficient samples), are invertible operators. This implies that the divisive normalization process itself does not inherently discard information about the original stimulus.
  • Broad Applicability of DNP Framework: The DNP framework is shown to be a unifying model, capable of describing a wide array of existing neural circuit models and phenomenological approaches in sensory processing.
  • Information Preservation: The ability to perfectly recover the input stimulus suggests that DNPs act as efficient coding mechanisms that transform sensory information without significant loss, which is crucial for downstream neural processing.
  • Sample Dependency: The fidelity of the recovery is directly dependent on the number of available output samples. More samples lead to better reconstruction.
  • Kernel Order Matters: The complexity of the recovery algorithm and the properties of the recovered components (e.g., rank-1 for second-order kernels) depend on the order of the kernels used in the DNP model.

Key Results

  • Perfect Recovery Achieved: The paper demonstrates both theoretically and computationally that perfect input recovery is possible for DNPs.
  • High SNR/PSNR: For stimuli processed by DNPs with first-order kernels, recovery reaches Signal-to-Noise Ratios above 133 dB, indicating near-perfect reconstruction.
  • Rank-1 Property for Second-Order Kernels: For DNPs with second-order kernels, perfect recovery is achieved when the recovered second-order input coefficient U has rank 1, a property that is maintained by their proposed rank-minimization algorithm.
  • Sample Threshold: Perfect recovery can always be achieved when the number of samples exceeds a certain threshold, which is illustrated in Figure 3.

Limitations and Future Work

  • Sufficient Samples: The primary condition for perfect recovery is having a "sufficient number of samples." The paper doesn't explicitly define what constitutes "sufficient" for all scenarios, though it provides empirical evidence and theoretical bounds.
  • Noise and Real-World Data: The current analysis primarily focuses on ideal conditions. Future work could explore the robustness of these recovery methods in the presence of noise and their application to real-world, noisy sensory data.
  • Complexity of Higher-Order Kernels: While the paper addresses first and second-order kernels, extending the invertible recovery framework to DNPs with even higher-order kernels could be a future direction.
  • Biological Plausibility: While DNPs are inspired by biological circuits, further investigation into the biological plausibility of the proposed recovery algorithms might be warranted.

Summary updated

Abstract

Abstract Divisive normalization is a canonical neural computation employed for sensory adaptation in vision, olfaction and attention modulation. While Divisive Normalization has been proposed to be an efficient coding algorithm, it remains unclear whether such transformation results in information loss in a dynamic setting. Leveraging a previously proposed general mathematical framework called the Divisive Normalization Processor (DNP), we first show that the DNP circuit describes a wide class of neural circuits as well as phenomenological models including the Linear-Nonlinear cascade model. We then demonstrate both theoretically and computationally that the DNP is an invertible operator that could faithfully represent input information given sufficient output samples.
BibTeX
@misc{lazar2022divisive,
  author = {Aurel A. Lazar and Tingkai Liu and Yiyin Zhou},
  title = {{Divisive Normalization Circuits Faithfully Represent Auditory and Visual Sensory Stimuli}},
  year = {2022},
  doi = {10.1101/2022.09.17.506431},
  howpublished = {bioRxiv},
}
Figure