Introduction
In sequential learning, neural networks frequently suffer from catastrophic forgetting, where learning a new task overwrites the weights necessary for previous ones. Recently, Kolmogorov-Arnold Networks (KANs) have gained significant attention as a potential solution. Unlike traditional Multilayer Perceptrons (MLPs) that use fixed linear weights, KANs utilize learnable 1D B-splines. Because B-splines are "local"—meaning changes to a coefficient only affect a small portion of the curve—early experiments suggested KANs might inherently resist forgetting through "local plasticity."
However, this paper challenges that narrative, demonstrating that while KANs excel in simple scenarios with disjoint input spaces (as seen in Figure 1), they still suffer from severe forgetting in more complex, real-world tasks where features overlap.
The "Spline Input Overlap" Trap
The authors identify three structural reasons why KANs fail to maintain knowledge in continual learning settings:
- Input Overlap: Local plasticity only prevents forgetting if tasks occupy different regions of the input space. If Task A and Task B both provide inputs in the same range (e.g., -0.5 to 0.5), the optimizer inevitably overwrites the same spline coefficients.
- Width-Sparsity Trade-off: For a fixed parameter count, KANs are structurally "narrower" than MLPs. This leads to denser gradients, whereas wider MLPs benefit from sparser updates that naturally reduce the likelihood of overwriting important weights.
- The Global Scale Problem: Standard KANs use a global scale parameter ($w_{j,i}$) for each spline. Adjusting this single scalar to optimize a new task shifts the entire spline, erasing previously learned information across all regions.
WiseKAN: A Specialized Isolation Algorithm
To address these failures, the authors introduce WiseKAN, a parameter isolation method tailored to KAN's unique architecture. WiseKAN employs several key strategies:
- Visibility & Freezing: An allocation mask dedicates specific parameters to specific tasks. Past parameters remain "visible" for computation but are "frozen" (gradients masked to zero) to prevent modification.
- Hierarchical Pruning: Leveraging KAN's hierarchy, WiseKAN first removes redundant splines and then prunes individual coefficients within active splines, efficiently freeing up capacity for future tasks.
Key Results and Insights
The study evaluated WiseKAN against standard MLPs and CL-augmented variants (using EWC and SI) on benchmarks like TI Feynman and TI Europe Wind Farm.
As shown in Figure 6, "naked" KANs fail significantly when tasks overlap (indicated by label F). In contrast, WiseKAN consistently outperformed both standard MLPs and other isolation methods like PackNet. The results suggest that when forgetting is explicitly controlled, the high expressivity of KANs provides a significant performance edge over traditional architectures.
Conclusion
While KANs are not a "silver bullet" for catastrophic forgetting, they offer a powerful foundation for continual learning when paired with proper isolation techniques like WiseKAN. Future work remains to scale these benefits to class-incremental settings where task IDs are not provided, and to determine if the performance gains justify the increased computational complexity compared to well-tuned MLPs.
Summary updated