AdvancedAdvanced Neural Architectures

Kolmogorov–Arnold Networks (KAN)

Networks that place learnable activation functions (B-spline curves) on the edges instead of fixed weights, while the nodes only sum their inputs, inspired by the Kolmogorov–Arnold representation theorem.

Overview

NameKolmogorov–Arnold Networks
Year introduced2024
ResearchersZiming Liu, Yixuan Wang, Sachin Vaidya, Fabian Ruehle, James Halverson, Marin Soljačić, Thomas Y. Hou, Max Tegmark
InstitutionMIT et al.
Original paperKAN: Kolmogorov–Arnold networks arXiv
Data typeTabular data, Physical fields and functions
DomainsTabular data, Scientific computing
Computational costHigher than an MLP per parameter because of computing the basis functions.
Data sizeScientific problems and small low-dimensional datasets.

Problem it solves

In an MLP the activation functions are fixed on the nodes and the weights are linear, which makes the learned functions hard to interpret. The authors sought models that are more interpretable and accurate on low-dimensional scientific problems.

Background and evolution

What came before: Multilayer Perceptron, Kolmogorov–Arnold theorem (1957)

  1. 1957

    Kolmogorov published the theorem representing any continuous multivariate function as a composition of univariate functions and addition. [ref]

  2. 2024

    KAN was posted on arXiv in April 2024, then published at ICLR 2025. [ref]

Core idea

The theorem states that $f(x_1,\dots,x_n)=\sum_q\Phi_q\left(\sum_p\phi_{q,p}(x_p)\right)$. KAN generalises this to deeper layers: each edge carries a univariate function $\phi(x)$ represented as a sum of B-spline basis functions with learned coefficients (plus a SiLU base function), and each node sums what reaches it. Each edge function can be plotted and matched against a symbolic formula.

Example: To learn $f(x,y)=\exp(\sin(\pi x)+y^2)$, a KAN of shape [2,1,1] can learn edges resembling $\sin(\pi x)$ and $y^2$ and then $\exp$, so the symbolic structure shows up in the plots.

Interactive diagram

A simplified illustrative diagram with the dimensions of a small example, not a drawing matching every detail of the original implementation. Click any layer to see its role, dimensions and mathematical operation.

Select a layer in the diagram.

Components

ComponentRole
InputLow-dimensional numeric variables
Learnable edge functions$\phi(x)=w_b\,\mathrm{silu}(x)+w_s\sum_i c_iB_i(x)$
Spline gridA grid of G intervals and degree k (usually 3)
Node summationThe node only sums its inputs
Grid extensionRefining the grid during training
Pruning / Symbolic regressionPruning edges and matching functions to symbols

Equations

1. Kolmogorov–Arnold representation theorem

$$f(x_1,\dots,x_n)=\sum_{q=1}^{2n+1}\Phi_q\!\left(\sum_{p=1}^{n}\phi_{q,p}(x_p)\right)$$

Symbols
$\phi_{q,p}$Inner univariate functions
$\Phi_q$Outer univariate functions
Dimensions
For $n$ variables, $2n+1$ outer functions are enough
Numerical example
For $n=2$, 5 outer terms are enough.
Relation to the architecture
The theoretical basis, noting that the functions in the theorem may be highly non-smooth.

2. KAN layer

$$x_{l+1,j}=\sum_{i=1}^{n_l}\phi_{l,j,i}(x_{l,i}),\qquad \phi(x)=w_b\,\mathrm{silu}(x)+w_s\sum_{m=1}^{G+k}c_m B_m(x)$$

Symbols
$B_m$B-spline basis functions
$G$Number of grid intervals
$k$Spline degree
$c_m$Learned coefficients
Dimensions
About $n_l\,n_{l+1}(G+k+2)$ parameters per layer
Numerical example
A [2→5] layer with $G=5,k=3$: each edge has 8 spline coefficients + two weights ⇒ $10\times10=100$ parameters.
Relation to the architecture
More parameters per edge than an MLP, but the network can be smaller.

3. B-spline functions (Cox–de Boor recursion)

$$B_{i,0}(x)=\mathbf{1}[t_i\le x<t_{i+1}],\quad B_{i,k}(x)=\frac{x-t_i}{t_{i+k}-t_i}B_{i,k-1}(x)+\frac{t_{i+k+1}-x}{t_{i+k+1}-t_{i+1}}B_{i+1,k-1}(x)$$

Symbols
$t_i$Grid knots
Dimensions
A scalar for each $x$
Numerical example
With uniform knots and a step of 1, $B_{i,1}$ is a triangular "hat" with a peak of 1.
Relation to the architecture
They make the function local: changing a coefficient affects only a small part of the domain.

How it works, step by step

  1. A spline grid is set for each edge over the input range.
  2. The basis function values are computed for each input.
  3. Each edge function is computed and then summed at the nodes.
  4. The layers repeat.
  5. The coefficients are trained by gradient descent with a regulariser for pruning.
  6. The grid can be refined and the functions matched to symbolic formulas.

Strengths and limitations

Strengths

  • Higher interpretability on low-dimensional problems.
  • Good accuracy in approximating smooth functions and scientific equations, according to the authors' experiments.
  • Symbolic formulas can be extracted.

Limitations

  • Slower to train than an MLP with the same parameters in current implementations.
  • The advantage over MLP is not general; later studies found mixed results depending on the task.
  • Still a recent research architecture.

When to use it, and when not?

Good for

  • Discovering symbolic relationships in scientific data.
  • Approximating smooth low-dimensional functions when interpretability is wanted.

Not suited for

  • High-dimensional images and text without combining it with other architectures.
  • When speed is a priority.

Applications

DomainUse
Scientific computingApproximating solutions of equations and discovering symbolic laws.
Tabular dataInterpretable models on small tabular data.

PyTorch code

A simplified teaching code sample that runs on small synthetic data, covering: imports, model definition, data, forward pass, loss, optimizer, training loop, evaluation and inference. It is not the paper's full implementation.

The authors' official implementation is the pykan library; this code is educational and shows a KAN layer with B-spline functions without grid refinement, pruning or symbolic matching.

Python (PyTorch)
# Kolmogorov–Arnold Network (KAN) with B-splines — an educational implementation
# A simplified educational example in PyTorch (not the paper's full original implementation): small synthetic data,
# runs on the CPU in seconds. The numbers it prints belong to this example only.
# 1) Import the libraries
import torch
import torch.nn as nn
import torch.nn.functional as F

torch.manual_seed(0)

# 2) Define the model
class KANLayer(nn.Module):
    def __init__(self, n_in, n_out, G=5, k=3, lo=-1.0, hi=1.0):
        super().__init__()
        h = (hi - lo) / G
        self.register_buffer('t', torch.arange(-k, G + k + 1) * h + lo)      # extended knots
        self.k = k
        self.c = nn.Parameter(torch.randn(n_out, n_in, G + k) * 0.1)        # spline coefficients
        self.wb = nn.Parameter(torch.ones(n_out, n_in) * 0.5)               # silu base weight
    def bsplines(self, x):                                                  # x: B×n_in → B×n_in×(G+k)
        x = x.unsqueeze(-1); t = self.t
        Bk = ((x >= t[:-1]) & (x < t[1:])).float()
        for p in range(1, self.k + 1):
            Bk = (x - t[:-(p + 1)]) / (t[p:-1] - t[:-(p + 1)]) * Bk[..., :-1] + \
                 (t[p + 1:] - x) / (t[p + 1:] - t[1:-p]) * Bk[..., 1:]
        return Bk
    def forward(self, x):
        spline = torch.einsum('bim,oim->bo', self.bsplines(x), self.c)       # Σ_i Σ_m c B(x_i)
        base = F.silu(x) @ self.wb.T
        return base + spline

class KAN(nn.Module):
    def __init__(self, width=(2, 5, 1), G=5):
        super().__init__()
        self.layers = nn.ModuleList([KANLayer(a, b, G) for a, b in zip(width[:-1], width[1:])])
    def forward(self, x):
        for i, l in enumerate(self.layers):
            x = l(x)
            if i < len(self.layers) - 1:
                x = torch.tanh(x)            # keep the inputs inside the grid range [-1, 1] (an educational simplification)
        return x

# 3) Prepare the data
# f(x, y) = exp(sin(πx) + y²) normalised, x, y ∈ [-1, 1]
def f(X):
    return torch.exp(torch.sin(torch.pi * X[:, :1]) + X[:, 1:] ** 2) / 3.0
X = torch.rand(1000, 2) * 2 - 1
Y = f(X)
X_train, y_train, X_test, y_test = X[:800], Y[:800], X[800:], Y[800:]
model = KAN()

# 4) Loss function and optimizer
loss_fn = nn.MSELoss()
opt = torch.optim.Adam(model.parameters(), lr=2e-2)

# 5) Training loop with the forward pass
EPOCHS, BATCH = 60, 64
for epoch in range(EPOCHS):
    model.train()
    perm = torch.randperm(len(X_train))
    total = 0.0
    for i in range(0, len(X_train), BATCH):
        idx = perm[i:i + BATCH]
        xb, yb = X_train[idx], y_train[idx]
        out = model(xb)                       # Forward Pass
        loss = loss_fn(out, yb)
        opt.zero_grad()
        loss.backward()                    # Backpropagation
        opt.step()
        total += loss.item() * len(idx)
    if epoch % max(1, EPOCHS // 4) == 0 or epoch == EPOCHS - 1:
        print(f"epoch {epoch:3d}  loss={total / len(X_train):.4f}")

# 6) Evaluation
model.eval()
with torch.no_grad():
    xb = X_test
    out = model(xb)
    mse = F.mse_loss(out, y_test).item()
    print(f"test MSE = {mse:.4f}")

# 7) Inference on a new sample
with torch.no_grad():
    xb = X_test[:1]
    out = model(xb)
    print("prediction:", out.squeeze().tolist())

This code was run on 2026-11-20 in 3.8 seconds (PyTorch (CPU), Python 3 – local build machine).

Sample output:

test MSE = 0.0000

Trainable parameters in the example: model = 135.

These results are for small synthetic data and few epochs, and only prove that the code runs; do not compare them with published results.

Try it and go deeper

Related architectures

Beginner1986

Multilayer Perceptron

Multilayer Perceptron (MLP)

A network of successive linear layers separated by nonlinear activation functions; it is the first building block on which most modern architectures are built.

Foundations

Intermediate2019

Physics-Informed Neural Networks

Physics-Informed Neural Networks (PINN)

A network that approximates the solution of a partial differential equation u_θ(x,t) and is trained with a loss that combines the equation's residual (computed by automa…

Physics-Informed Neural Networks

Research2023

Neural Operators

Neural Operators (NO)

A family of architectures that learn operators between function spaces rather than between fixed-size vectors, by stacking pointwise linear layers with integral operator…

Physics-Informed Neural Networks

Research ideas

  • Compare KAN with an MLP at the same parameter count on PDE problems, measuring time and accuracy.
  • Use KAN as an interpretable classification head on top of CNN features for medical images.

Starting suggestions, not confirmed research gaps; review the recent literature before adopting them.

Test your understanding

  1. Concept

    Where are the learnable functions in KAN?

  2. Concept

    What theoretical basis inspired KAN?

  3. Comparison

    Which statement about comparing KAN with MLP is most accurate?

References

  1. Liu, Z., Wang, Y., Vaidya, S., Ruehle, F., Halverson, J., Soljačić, M., Hou, T. Y., & Tegmark, M. (2025). KAN: Kolmogorov–Arnold networks. International Conference on Learning Representations (ICLR). https://arxiv.org/abs/2404.19756

    arXivAutomatically verified via OpenAlex/arXiv – 2026-11-20

  2. Kolmogorov, A. N. (1957). On the representation of continuous functions of many variables by superposition of continuous functions of one variable and addition. Doklady Akademii Nauk SSSR, 114(5), 953–956 (in Russian). English translation: American Mathematical Society Translations, Series 2, 28, 55–59 (1963). https://doi.org/10.1090/trans2/028/04

    DOIAutomatically verified via Crossref – 2026-11-20DOI of the 1963 English translation

  3. Cybenko, G. (1989). Approximation by superpositions of a sigmoidal function. Mathematics of Control, Signals and Systems, 2(4), 303–314. https://doi.org/10.1007/BF02551274

    DOIAutomatically verified via Crossref – 2026-11-20