Plastic Asymmetric Lateral Inhibition for Noise-Robust Sensory Coding

Lateral inhibition is a wiring pattern in which active units suppress their neighbours. In sensory systems and in several deep networks it makes overlapping inputs look more different, which helps a simple downstream classifier. Its known weakness is that it is hard-wired. In a spiking model of the fruit-fly olfactory system, fixed lateral inhibition raises odor classification from about 92% to about 98% when inputs are clean. The same circuit collapses to about 54% at noise intensity 0.3, because it suppresses weak true signals along with the noise (Seemingly Redundant Modules Enhance Robust Odor Learning in Fruit Flies). A separate literature on inhibitory synaptic plasticity shows that inhibitory connections can learn. They can form "negative images" that cancel predictable input and keep a network stable (Inhibitory Plasticity: Balance, Control, and Codependence, Inhibitory engrams in perception and memory). Nobody has put such a rule inside the lateral inhibitory layer of a sensory classifier and asked whether it prevents this collapse.

We propose making the lateral inhibitory weights plastic under a local, unsupervised rule. The rule strengthens inhibition between channels that co-vary in their activity and pushes each unit toward a target firing rate. A competition term, of the kind that limits how much total input a neuron can receive, forces the resulting connectivity to be asymmetric. The claim is that suppression learned this way lands on the structure that odors share, the redundancy that lateral inhibition exists to remove. Suppression should pull back from channels where independent noise dominates, because independent noise raises a channel's variance but not its correlation with neighbours. If the claim holds, the network keeps the clean-condition gain of lateral inhibition without paying the high-noise penalty.

Core Claim

A fast, covariance-driven, competitively normalized inhibitory rule on the lateral connections should produce three things. First, it should yield asymmetric, non-local inhibition that tracks the input's correlation structure. Second, it should recover clean accuracy near the fixed-inhibition level of about 98% on the Li et al. benchmark. Third, it should beat both fixed inhibition (about 54%) and spike-frequency adaptation (about 69%) at noise intensity 0.3. Spike-frequency adaptation is a neuron's tendency to fire less under sustained input.

Crucially, the gain must come from the structure of the learned connectivity and not merely from turning overall inhibition down. That distinction is what separates this proposal from a tuned scalar knob, and the experiments are built around it.

Relation to Prior Work

Three papers motivate the gap. Li et al. found the high-noise collapse, but their inhibitory wiring is fixed from anatomy and only the readout is trained, with backpropagation through time. Shlizerman et al. inferred from recordings that insect antennal-lobe inhibition is asymmetric and non-local and that this stabilizes codes. Their connectivity was fit to data, not learned by the circuit. We ask whether that asymmetry can emerge from a local rule.

The closest existing work is Sparse Coding and Lateral Inhibition Arising from Balanced and Unbalanced Dendrodendritic Excitation and Inhibition. It lets a use-dependent rule shape reciprocal inhibition in a detailed olfactory-bulb model. The result is that strongly driven cells end up balanced while weakly driven neighbours become inhibition-dominated, with decorrelation peaking near a 6.5:1 inhibition-to-excitation ratio. That work reports decorrelation and sparseness on 72 odors. It never measures classification or noise robustness, and its rule is phenomenological. We keep its central intuition, that plasticity should give different neurons different balance regimes, and test it as a robustness mechanism.

Two papers supply the method. Synapse-type-specific competitive Hebbian learning forms functional recurrent networks shows that letting inhibitory synapses compete for a limited resource pool yields surround suppression and decorrelated tuning. It does so on synthetic gratings, with homogeneous pools and no task. We borrow its normalization and apply it per postsynaptic unit, which is where asymmetry comes from. Synaptic Plasticity in Neural Networks Needs Homeostasis with a Fast Rate Detector constrains the design: the rate-tracking term must be fast, below roughly 25 s in their simulations, or the network runs away.

One paper sits closer to engineering. Training Deep Normalization-Free Spiking Neural Networks with Lateral Inhibition uses lateral excitation-inhibition circuits in deep spiking networks, but trains them end to end with gradient scaling. It therefore tells us nothing about label-free adaptation to a shifted noise level.

Research Program

We reimplement the Li et al. fly model. It has projection neurons with lateral inhibition from local interneurons, a Kenyon-cell expansion layer, and a readout trained with surrogate-gradient backpropagation through time, exactly as in their setup. We replace the fixed lateral matrix $W$ with plastic weights updated by the rule below.

$$\Delta W_{ij} = \eta\,\big[(r_i-\bar r_i)(r_j-\bar r_j) + \lambda\, r_j\,(r_i-\rho_0)\big], \qquad W_{i\cdot}\leftarrow \frac{W_{i\cdot}}{\sum_j W_{ij}}\,B_i$$

The symbols are defined as follows. $W_{ij}\ge 0$ is inhibition from unit $j$ onto unit $i$, and $r_i$ is unit $i$'s firing rate. $\bar r_i$ is a running average of $r_i$ whose time constant respects the fast-homeostasis bound. $\rho_0$ is a target rate and $\lambda$ weights the homeostatic term. $B_i$ is unit $i$'s inhibitory resource budget, and the second expression renormalizes each unit's incoming weights to that budget. Row-wise normalization makes $W$ asymmetric whenever units differ in how many correlated partners they have.

We deliberately keep the budget adaptive: $B_i$ shrinks when unit $i$'s signal-to-noise estimate falls. This is the one design choice that directly targets the "crushes weak signals" failure. We ablate it explicitly.

The lateral weights learn online and without labels, both during training and at test time. The readout is trained once, on clean or mildly noisy data. Beyond accuracy, we analyse the learned matrix itself. We measure asymmetry as the norm of the antisymmetric part of $W$ over the norm of $W$, test whether $W$ correlates with the signal covariance versus the noise covariance, and measure the per-unit inhibition-to-excitation ratio. That ratio lets us check whether the balanced-versus-inhibition-dominated split of Yu et al. appears.

Baselines

The naive reference point is the Li et al. network without lateral inhibition. It reaches about 92% on 1,000 odor classes in the clean condition, as reported in Seemingly Redundant Modules Enhance Robust Odor Learning in Fruit Flies. The fixed-inhibition model from the same paper is the direct competitor, reported at about 98% clean and about 54% at noise 0.3.

The strongest existing method in the file for high noise is the spike-frequency-adaptation variant from the same paper, reported at about 69% at noise 0.3. We also run adaptation combined with fixed inhibition. That combination is the paper's own suggestion for handling both regimes, and plastic inhibition has to beat it or complement it.

Two further baselines guard against a trivial explanation. The first is an oracle scalar gain on the fixed matrix, tuned separately at every noise level on a validation split. The second is the full lateral matrix trained end to end by backpropagation through time, with labels, at each noise level. That supervised matrix is an upper bound, not a rival.

Our key ablations remove one ingredient at a time: the covariance term, leaving only rate homeostasis; the row normalization, which yields symmetric inhibition; the adaptive budget; and the fast time constant, replacing it with one well above the 25 s bound.

Every variant uses the same odor sets, noise generators (Gaussian and Ornstein–Uhlenbeck), readout architecture, number of readout training epochs, and seeds. Each gets an equal hyperparameter search budget of the same number of configurations. Beating the competitors means exceeding 69% at noise 0.3 while staying within one point of 98% clean. On top of that, the method must exceed the oracle scalar at a noise level it was not tuned on, using the same readout with no retuning.

Why not just…?

Why not just scale fixed inhibition down when noise is high? This is the most serious threat. Much of the collapse may simply be over-inhibition. The oracle-scalar baseline settles the question. If plastic inhibition only matches the oracle at the tuned noise levels, the honest conclusion is narrower: the contribution is autonomous, label-free selection of that scalar. The structural claim then fails. The structural claim survives only if plastic inhibition wins at held-out noise levels, or on the shared-noise variant described below, where no single gain can separate signal from noise.

Why not just use spike-frequency adaptation, which already reaches about 69%? Li et al. show adaptation acts as a temporal high-pass filter. It helps under noise but gives up the spatial contrast that lateral inhibition provides in clean conditions. We run adaptation alone and adaptation plus plastic inhibition. If the combination beats adaptation plus fixed inhibition, the two mechanisms are complementary and plastic inhibition is not redundant.

Why not just learn the lateral matrix with backpropagation? Gradient-trained inhibition, as in DeepEISNN, needs labels at the noise level of deployment. The supervised baseline measures how much accuracy the local rule leaves on the table. The main test is a shift in noise level at test time with no labels, where only an unsupervised rule can adapt. If the supervised matrix trained on clean data transfers as well as our online rule, the adaptation argument fails and we will say so.

Proposed Experiments

The primary experiment sweeps noise intensity from 0 to 0.5 on 1,000 and 10,000 synthetic odor classes, under both Gaussian and Ornstein–Uhlenbeck noise, with 10 seeds. We report accuracy curves for every baseline and ablation.

The decisive check combines three results: plastic inhibition must hold at least 69% at noise 0.3, stay near 98% clean, and exceed the oracle scalar at held-out noise levels. We add one controlled extension that isolates the negative-image mechanism. In it, a fraction of the noise is shared across channels. The covariance term should learn to cancel this shared mode, whereas no scalar gain can do so.

The structural predictions are also falsifiable. Learned asymmetry should grow as the input correlations become more heterogeneous. With row normalization removed, asymmetry should vanish and robustness should drop. With the time constant raised above the fast-homeostasis bound, rates should become unstable, as Zenke et al. predict.

As a secondary transfer test, we would make the pull kernel plastic in the multi-scale push-pull network on CIFAR-10-C. The push-pull network pairs each feature filter with a larger opposite-sign "pull" kernel that subtracts surrounding activity. Multi-Scale Unrectified Push-Pull with Channel Attention for Enhanced Corruption Robustness reports 70.97% average corruption accuracy with ResNet-20 for its multi-scale variant and 66.27% on noise corruptions for its attention variant. We would compare against those figures under the same backbone and no corruption augmentation. This test is conditional on the olfactory result: if plastic inhibition does not beat the oracle scalar there, we will not claim it for vision.