KAConvNet: Kolmogorov–Arnold Convolutional Networks for Vision Recognition

Zhaoxiang Liu, Zhicheng Ma, Kaikai Zhao, Kai Wang, Shiguo Lian

TL;DR

This paper introduces KAConvNet, a novel convolutional neural network architecture that integrates the Kolmogorov-Arnold (KA) representation theorem. By developing a new KAConvLayer, the authors achieve competitive performance in vision tasks like image classification, object detection, and semantic segmentation, while offering improved interpretability and potentially faster training compared to existing KA-based methods.

What Problem Does This Paper Address?

Convolutional Neural Networks (CNNs) have been the workhorse of computer vision for years. Recently, Kolmogorov-Arnold Neural Networks (KANs) have emerged as a promising alternative to traditional Multi-Layer Perceptrons (MLPs), offering fewer parameters and greater explainability. However, integrating the theoretical underpinnings of KANs with the established success of CNNs for vision tasks has been limited. Existing attempts often simplify KANs by replacing learnable activations with weights or using computationally inefficient B-spline curves, which can lead to overfitting. This paper aims to bridge this gap by proposing a novel convolutional layer that deeply integrates the KA representation theorem, addressing the limitations of prior work and unlocking the potential of KANs in CNN architectures for vision recognition.

Methods

The core innovation is the Kolmogorov-Arnold Convolution Layer (KAConvLayer). This layer is designed to deeply integrate the KA representation theorem with convolution. It applies the inner functions of the theorem to aggregate information within each channel and the outer function across channels for effective information exchange. This design allows for distinct functional mappings per channel while reducing parameters.

Instead of B-splines, the authors propose a piecewise linear learnable activation function called GLinear. This function is chosen for its lower computational complexity and reduced risk of overfitting compared to B-splines. The KAConvLayer uses this GLinear function on weights to represent external functions and convolution with learnable activations for internal functions.

The paper then proposes KAConvNet, a pure CNN architecture built upon these KAConvLayers. It follows a standard multi-stage design (Stem, Stages, Transitions) similar to popular CNNs, incorporating KAConvBlocks that utilize KAConvLayers and channel attention (SE Block).

Datasets

The paper evaluates KAConvNet on the following standard datasets:

  • ImageNet-1K: For image classification.
  • MS COCO: For object detection.
  • Cityscapes: For semantic segmentation.
  • CIFAR-100: Used for ablation studies to analyze the impact of different activation functions and layer placements.

Key Insights and Ideas

  • Deep Integration of KA Theorem: The paper moves beyond simply replacing components of existing architectures with KAN elements. Instead, it deeply integrates the KA representation theorem into the convolutional operation itself, leading to a more theoretically grounded and potentially more effective approach.
  • GLinear for Efficiency and Robustness: The choice of the GLinear function over B-splines is a key insight. It addresses the computational inefficiency and overfitting issues associated with B-splines, making the proposed KAConvNet more practical for vision tasks.
  • Channel-wise Functional Mapping: The KAConvLayer's design allows each channel to learn its own distinct functional mapping, which is crucial for capturing diverse features in images, unlike some prior KA-based methods that used shared activations.
  • Interpretability Boost: By grounding the convolutional layer in a mathematical theorem and using learnable activation functions on weights, the authors claim improved method interpretability.
  • Scalability and Versatility: KAConvNet demonstrates strong performance across classification, detection, and segmentation, suggesting its versatility as a backbone architecture.

Key Results

  • Image Classification: KAConvNet-L achieves a Top-1 accuracy of 80.1% on ImageNet-1K, competitive with mainstream Mamba-based models, CNNs, and ViTs, while using fewer parameters than some comparable models. KAConvNet-B and KAConvNet-S also show strong performance relative to their parameter counts.
  • Object Detection: KAConvNet variants achieve competitive mAP scores on MS COCO, outperforming or matching other CNN and Transformer-based backbones.
  • Semantic Segmentation: KAConvNet-L achieves the best performance on Cityscapes with a Mean IoU of 70.58 and Mean Pixel Accuracy of 79.51, outperforming established baselines like ResNet-18.
  • Inference Speed: Tab. 7 shows that KAConvLayer achieves superior inference speed compared to other KA-based convolution layers, attributed to the GLinear function and channel-wise processing.
  • Ablation Studies: Experiments on CIFAR-100 indicate that B-spline functions lead to lower accuracy, likely due to overfitting, while GLinear functions perform well. Placing KAConvLayers in shallower layers of a VGG-like network yields better results when residual connections are not used.

Limitations and Future Work

  • Computational Efficiency: While KAConvNet is more efficient than some KA-based methods, it is acknowledged that learnable activation functions for feature maps still increase computation compared to standard convolutions. The authors mention that the computational efficiency is "somewhat lower than that of the standard convolution."
  • Kernel Size: The paper notes that KAConvNet, with its default small kernel size (3x3), slightly lags behind existing large-kernel convolution networks in capturing long-range pixel dependencies.
  • Future Work: The authors suggest increasing the kernel size of KAConvLayers (e.g., to 5x5) to improve long-range dependency capture and explore computational acceleration techniques for KA convolutions. They also advocate for further research into architectural designs and mathematical theorems to match the efficiency of leading ViTs and CNNs.

Summary updated

BibTeX
@article{liu2026kaconvnet,
  author = {Zhaoxiang Liu and Zhicheng Ma and Kaikai Zhao and Kai Wang and Shiguo Lian},
  title = {{KAConvNet: Kolmogorov–Arnold Convolutional Networks for Vision Recognition}},
  journal = {Image and Vision Computing},
  year = {2026},
  doi = {10.1016/j.imavis.2026.105983},
}
Figure