For decades, the Multi-Layer Perceptron (MLP) has served as the foundational building block of deep learning. The structural formula of an MLP is deeply familiar: a node receives a weighted sum of inputs from the previous layer, passes that sum through a fixed non-linear activation function (such as ReLU or Sigmoid), and transmits the result forward. To learn increasingly complex functions, an MLP relies on architectural depth—stacking layers to progressively slice high-dimensional space with flat hyperplanes.
A rapidly growing body of research explores a fundamental alternative to this paradigm: the Kolmogorov-Arnold Network (KAN). As formalized by Ziming Liu et al. in KAN: Kolmogorov–Arnold Networks, KANs completely invert the traditional allocation of labor. Instead of fixed activation functions at the nodes and scalar weights on the edges, KANs place learnable univariate activation functions directly on the edges. The nodes themselves perform no non-linear transformations; they simply sum the incoming signals.
Traditional MLP Node:
Inputs (x₁, x₂) ──> [ Weights (w₁, w₂) ] ──> [ Sum (∑) ] ──> [ Fixed Activation (σ) ] ──> Output
Kolmogorov-Arnold Network (KAN) Node:
Inputs (x₁, x₂) ──> [ Learnable Functions (f₁(x₁), f₂(x₂)) ] ──> [ Sum (∑) ] ──> Output
By shifting the complexity of function approximation from static node hierarchies to flexible, edge-based transformations, this architecture introduces a completely different inductive bias—one that yields remarkable parameter efficiency, mathematical interpretability, and precision in scientific modeling.
The Intuition: Why Edge-Based Functions Work
The theoretical foundation of this architecture traces back to the Kolmogorov-Arnold representation theorem. The theorem states that any multivariate continuous function can be decomposed into a finite composition of continuous univariate functions and addition.
To understand why this is powerful, consider how a standard MLP approximates a smooth curve. A ReLU-based network must piece together thousands of tiny linear segments, requiring significant depth and parameter counts to minimize error. If you want to increase the precision of an MLP, you must add more layers or wider hidden dimensions, which scales the parameter count aggressively.
In contrast, a KAN parameterizes each edge function using flexible, localized curves—most commonly B-splines, power units, or polynomials. This allows a single connection between two nodes to adaptively fit intricate non-linear relationships. Instead of expanding the entire network architecture to achieve higher precision, a KAN can undergo "grid refinement," making the underlying spline grids finer without altering the network's macro-structure.
The theoretical viability of this approach has been rigorously validated. For instance, Xianyang Zhang et al. demonstrated in Generalization Bounds and Model Complexity for Kolmogorov–Arnold Networks that a KAN's generalization complexity scales based on the smoothness and coefficients of its learnable edge functions rather than the sheer quantity of hidden nodes. Furthermore, Yixuan Wang et al. proved in On the expressiveness and spectral bias of KANs that while an MLP can theoretically map to an edge-based architecture, representing fine-grained spline functions within a standard MLP requires a parameter count that scales poorly with the resolution of the spline grid.
This concept builds upon a long history of adaptive activation functions. Early precursors explored giving individual neurons unique, learnable parameters. Matthew Hoffman in Learning Activation Functions to Improve Deep Neural Networks proposed optimizing sums of hinge functions per neuron via gradient descent, while Qinliang Su et al. in A Probabilistic Framework for Nonlinearities in Stochastic Neural Networks showed that allowing individual units to learn unique activation shapes significantly improves pattern capture compared to uniform layer-wide activations. KANs take this a step further by decentralizing the activation function entirely, moving it from the neuron to every individual connection.
Overcoming the Computational Bottleneck
Despite their expressive power, original KAN formulations suffer from a major practical drawback: they are slow. Modern hardware accelerators (GPUs) are highly optimized for dense, uniform matrix multiplications. Evaluating recursive B-splines independently across thousands of network edges breaks this parallelism, creating severe computational bottlenecks.
To bridge the gap between theoretical expressivity and hardware efficiency, researchers have introduced several optimized variants:
- Algorithmic Restructuring: The authors of PowerMLP: An Efficient Version of KAN (Ruichen Qiu et al.) bypassed the spline bottleneck by shifting learnable functional components back to the nodes using a non-iterative "Power ReLU" basis function. This allows the model to leverage standard matrix operations, achieving a 40x training speedup while retaining KAN-like expressivity.
- Alternative Basis Functions: Instead of computationally heavy B-splines, models can use alternative polynomial bases. Nizar Bouguila developed LuKAN: A Kolmogorov-Arnold Network Framework for 3D Human Motion Prediction using Lucas polynomials to model complex spatial-temporal dynamics with shallower structures. Similarly, Sicong LAI et al. utilized Taylor polynomials in STKAN: Kolmogorov-Arnold Networks for Spatio-Temporal Forecasting to replace traditional MLP blocks.
- Hardware-Conscious Implementations: In real-time signal processing, Yihang Feng et al. demonstrated in Machine Learning-Based WiFi Indoor Localization with FasterKAN: Optimizing Communication and Signal Accuracy that optimizing the underlying functional execution can yield up to an 8.2x inference speedup over standard convolutional architectures by prioritizing edge-learning flexibility over deep, fixed-node hierarchies.
Opening the Black Box: Built-In Interpretability
One of the most compelling reasons to shift activations to the edges is transparency. In a standard MLP, a feature's influence is obscured as it passes through a dense web of weights and shared activations. In a KAN, because each connection possesses its own isolated univariate function, the internal logic of the network can be directly inspected.
Because these edge functions are one-dimensional curves, they can be plotted and visualized. Over the course of training, many edge functions naturally flatten out or simplify, allowing researchers to prune unnecessary connections and even distill the remaining curves into explicit symbolic mathematical expressions.
Input Feature (x) ───[ Learned Curve: f(x) = sin(x) ]───> Output Node
This explicit representation has driven adoption across domains where understanding why a model makes a decision is as critical as the decision itself:
- Biological Discovery: Fan Xu developed scKAN: interpretable single-cell analysis for cell-type-specific gene discovery and drug repurposing via Kolmogorov-Arnold networks, demonstrating that learnable activation curves allow researchers to directly visualize gene contributions and identify disease drivers without relying on complex attention mechanisms.
- Neural Cognitive Diagnosis: In educational modeling, Shangshang Yang et al. designed Endowing Interpretability for Neural Cognitive Diagnosis by Efficient Kolmogorov-Arnold Networks to simplify networks into sparse, symbolic-like expressions, letting educators track exactly which knowledge concepts drive student performance.
- Hardware Security: Kota Yoshida et al. applied this architecture to side-channel analysis in Can KANs Do It? Toward Interpretable Deep Learning-based Side-channel Analysis, opening the "black box" of deep learning models used to detect cryptographic vulnerabilities.
- Multimodal Fusion: In sentiment analysis, M. X. Luo et al. leveraged edge-based architectures in Towards Explainable Fusion and Balanced Learning in Multimodal Sentiment Analysis to decompose complex cross-modal interactions (text, audio, video) into inspectable mathematical curves.
Expanding the Architecture Ecosystem
The paradigm of learnable edge activations is rapidly expanding beyond simple tabular function fitting into structured data domains, replacing MLPs within broader deep learning frameworks.
Graph Learning
Traditional Graph Neural Networks (GNNs) rely on MLPs to transform messages passed between neighboring nodes. By swapping MLPs for KANs, the authors of KAGNNs: Kolmogorov-Arnold Networks meet Graph Learning (Roman Bresson et al.) demonstrated a massive leap in graph regression tasks, as flexible edge functions excel at fitting the smooth, continuous multivariate regularities common in scientific graph data. To enforce geometric constraints, Ran Elbaz et al. introduced FS-KAN: Permutation Equivariant Kolmogorov-Arnold Networks via Function Sharing, sharing univariate functions across edges to achieve strict permutation equivariance. Additionally, Taoran Fang et al. developed KAA: Kolmogorov-Arnold Attention for Enhancing Attentive Graph Neural Networks, replacing linear attention scoring with KANs to achieve near-infinite expressive power in ranking neighbor importance.
Continuous Fields and Physics
Modeling physical systems requires capturing sharp gradients and continuous spaces. Jin Lee et al. developed KANO: Kolmogorov-Arnold Neural Operator to overcome the limitations of Fourier Neural Operators when dealing with position-dependent physics. For complex partial differential equations, Changhong Mou et al. utilized edge-based splines in Energy-Dissipative Evolutionary Kolmogorov-Arnold Networks for Complex PDE Systems to provide the localized flexibility and smoothness needed to represent sharp spatial gradients effectively.
Vision and Parameter-Efficient Fine-Tuning
KANs are also proving to be highly effective as lightweight adapters for massive vision models. Xuan Cai designed LoKi: Low-dimensional KAN for Efficient Fine-tuning Image Models, a bottleneck architecture that replaces fixed activations in standard adapters with learnable B-splines, achieving superior task generalization with minimal memory overhead. Similarly, Ruichao Hou et al. built KAN-SAM: Kolmogorov-Arnold Network Guided Segment Anything Model for RGB-T Salient Object Detection, updating less than 1% of total parameters by injecting thermal features through KAN adapters. For specialized segmentation tasks, Daniele Rege Cambrin et al. integrated KANs into a U-Net structure (KAN You See It? KANs and Sentinel for Effective and Explainable Crop Field Segmentation), improving boundary detection accuracy while cutting computational costs nearly in half.
By rethinking the fundamental structure of the neuron, this research direction demonstrates that the way a network learns non-linearities is just as critical as its architectural depth. Shifting activation functions to the edges provides a mathematically grounded path toward models that are not only smaller and more precise, but fundamentally open to human inspection.