A biologically-plausible alternative to backpropagation using pseudoinverse feedback
Lists
Only you see this
Abstract
Despite its successes in both practical machine learning and neural modeling, the backpropagation algorithm has long been considered biologically implausible (Crick, 1989). Previous solutions to this biological implausibility have proposed the existence of a separate, error feedback network, in which error at the final layer may be propagated backwards to earlier layers in a manner similar to backpropagation. However, biological evidence suggests that feedback connections in the cortex may function more similarly to an autoencoder, rather than being exclusively used as error feedback (Marino, 2020; Chen et al., 2024). Here, we attempt to unify these two paradigms by showing how autoencoder-like, inverse feedback connections may be used to minimize error throughout a feedforward neural network. Our proposed mechanism, Reciprocal Feedback, consists of two contributions: first we show how a modification of the Recirculation algorithm (Hinton & McClelland, 1988) is capable of learning the Moore-Penrose pseudoinverse of a pair of network weights. Then, we will show how, using a Newton-like method (Hildebrandt & Graves, 1927), locally-learned pseudoinverse feedback connections may be used to facilitate an alternative optimization method to traditional gradient descent - while alleviating the need to compute the weight transpose, or use direct feedback connections from the final layer. In the MNIST and CIFAR-10 classification tasks, our method obtains an asymptotic error similar to backpropagation, in fewer iterations than comparable biologically-plausible algorithms, such as Feedback Alignment (Lillicrap et al., 2014).
BibTeX
@misc{cameron2025biologically,
author = {Mia Cameron and Yusi Chen and Terrence Sejnowski},
title = {{A biologically-plausible alternative to backpropagation using pseudoinverse feedback}},
year = {2025},
url = {https://openreview.net/forum?id=B5Dj4EhZPP},
howpublished = {OpenReview},
note = {Submitted to ICLR 2025},
}Loading related content...