Abstract
Neural network research has historically focused on architectural innovations, such as Transformers or Convolutional Neural Networks, while the underlying neuron model has remained largely stagnant since the McCulloch-Pitts (MP) model of 1943. The recently proposed Flexible Transmitter (FT) neuron model offers a bio-plausible alternative that leverages complex-valued mathematics to mimic neurotrophic potentiation and depression. While FT Networks (FTNets) have shown empirical promise in processing temporal-spatial signals, their theoretical properties have remained unexplored. This paper establishes the first mathematical foundation for FTNets, proving they are universal approximators with the potential for exponential efficiency over standard real-valued networks. Furthermore, we demonstrate that the loss landscape of feedforward FTNets is devoid of suboptimal local minima under mild conditions, facilitating global optimization.
Introduction
In the landscape of deep learning, the fundamental building block—the neuron—has seen remarkably little evolution compared to the architectures that house them. Most modern networks still rely on the MP neuron, which simplifies biological signaling into a weighted sum followed by a non-linear activation. The Flexible Transmitter (FT) model was introduced to bridge this gap, incorporating complex-valued dynamics to better represent biological signal transmission.
Despite empirical successes, the lack of theoretical rigor has hindered the broader adoption and understanding of FTNets. This research addresses this void by investigating three critical areas: expressivity (Universal Approximation), efficiency (Approximation Complexity), and optimizability (Loss Landscape).
Universal Approximation
The authors first establish that FTNets possess the necessary expressivity to serve as general-purpose models. By employing measure theory and complex analysis, the paper proves two primary theorems:
- Feedforward FTNets (F-FTNet): A one-hidden-layer F-FTNet can approximate any continuous function on a compact set to arbitrary precision.
- Recurrent FTNets (R-FTNet): An R-FTNet can approximate any discrete-time open dynamical system.
These results confirm that FTNets are not merely specialized tools for niche tasks but are mathematically equivalent in potential to standard Feedforward Neural Networks (FNNs) and Recurrent Neural Networks (RNNs).
Approximation Complexity: The Case for Efficiency
The most significant contribution of this work lies in the comparison of "approximation complexity"—the number of neurons required to achieve a specific level of accuracy. The authors identify a "complexity separation" between FTNets and traditional real-valued networks.
Exponential Advantage
The paper proves that there exist specific target functions where standard FNNs or RNNs require an exponentially large number of hidden neurons to achieve a low approximation error. Specifically, for an input dimension $I$, a standard network might require a width of $\Omega(e^{cI})$. In contrast, an FTNet can approximate these same functions using only a polynomial number of neurons, $O(I^{15/4})$. This suggests that for certain classes of data, FTNets are fundamentally more efficient at representation.
Worst-Case Parity
Critically, this efficiency does not come at the cost of a lower bound. The authors prove a "safety net" theorem: in the worst-case scenario, any function that can be approximated by a standard FNN or RNN can also be approximated by an FTNet using the same order of magnitude of hidden neurons ($O(H)$). Thus, FTNets offer a significant potential upside with no theoretical downside in terms of parameter scaling.
The Optimization Landscape
A perennial challenge in deep learning is the presence of suboptimal local minima, which can trap gradient-based optimization algorithms. The authors analyze the empirical loss surface of F-FTNets using the properties of holomorphic (complex differentiable) functions.
They prove that for an F-FTNet utilizing a holomorphic, non-polynomial activation function (such as zReLU), the loss landscape is remarkably well-behaved. Under the condition that training samples are linearly independent, the paper demonstrates that every local minimum is a global minimum. If the network has not reached a loss of zero, a mathematical path always exists to further decrease the loss, implying that local search algorithms like Stochastic Gradient Descent (SGD) are theoretically capable of finding the global optimum.
Limitations and Future Outlook
While this work provides a robust theoretical starting point, it focuses primarily on shallow (one-hidden-layer) architectures. The mathematical behavior of deep FTNets—specifically how they transform feature spaces across multiple layers—remains an open question. Additionally, the proof regarding the absence of suboptimal local minima currently applies only to the feedforward variant (F-FTNet), leaving the landscape of Recurrent FTNets for future investigation.
By providing these proofs of universal approximation, exponential efficiency, and global optimizability, this research elevates the FT neuron from a bio-inspired heuristic to a mathematically grounded alternative for modern machine learning.
Summary updated