Kolmogorov–Arnold Networks (KAN)
Networks that place learnable activation functions (B-spline curves) on the edges instead of fixed weights, while the nodes only sum their inputs, inspired by the Kolmogorov–Arnold representation theorem.
Overview
| Name | Kolmogorov–Arnold Networks |
|---|---|
| Year introduced | 2024 |
| Researchers | Ziming Liu, Yixuan Wang, Sachin Vaidya, Fabian Ruehle, James Halverson, Marin Soljačić, Thomas Y. Hou, Max Tegmark |
| Institution | MIT et al. |
| Original paper | KAN: Kolmogorov–Arnold networks arXiv |
| Data type | Tabular data, Physical fields and functions |
| Domains | Tabular data, Scientific computing |
| Computational cost | Higher than an MLP per parameter because of computing the basis functions. |
| Data size | Scientific problems and small low-dimensional datasets. |
Problem it solves
In an MLP the activation functions are fixed on the nodes and the weights are linear, which makes the learned functions hard to interpret. The authors sought models that are more interpretable and accurate on low-dimensional scientific problems.
Background and evolution
What came before: Multilayer Perceptron, Kolmogorov–Arnold theorem (1957)
Core idea
The theorem states that $f(x_1,\dots,x_n)=\sum_q\Phi_q\left(\sum_p\phi_{q,p}(x_p)\right)$. KAN generalises this to deeper layers: each edge carries a univariate function $\phi(x)$ represented as a sum of B-spline basis functions with learned coefficients (plus a SiLU base function), and each node sums what reaches it. Each edge function can be plotted and matched against a symbolic formula.
Interactive diagram
A simplified illustrative diagram with the dimensions of a small example, not a drawing matching every detail of the original implementation. Click any layer to see its role, dimensions and mathematical operation.
Select a layer in the diagram.
Components
| Component | Role |
|---|---|
| Input | Low-dimensional numeric variables |
| Learnable edge functions | $\phi(x)=w_b\,\mathrm{silu}(x)+w_s\sum_i c_iB_i(x)$ |
| Spline grid | A grid of G intervals and degree k (usually 3) |
| Node summation | The node only sums its inputs |
| Grid extension | Refining the grid during training |
| Pruning / Symbolic regression | Pruning edges and matching functions to symbols |
Equations
1. Kolmogorov–Arnold representation theorem
| $\phi_{q,p}$ | Inner univariate functions |
| $\Phi_q$ | Outer univariate functions |
2. KAN layer
| $B_m$ | B-spline basis functions |
| $G$ | Number of grid intervals |
| $k$ | Spline degree |
| $c_m$ | Learned coefficients |
3. B-spline functions (Cox–de Boor recursion)
| $t_i$ | Grid knots |
How it works, step by step
- A spline grid is set for each edge over the input range.
- The basis function values are computed for each input.
- Each edge function is computed and then summed at the nodes.
- The layers repeat.
- The coefficients are trained by gradient descent with a regulariser for pruning.
- The grid can be refined and the functions matched to symbolic formulas.
Strengths and limitations
Strengths
- Higher interpretability on low-dimensional problems.
- Good accuracy in approximating smooth functions and scientific equations, according to the authors' experiments.
- Symbolic formulas can be extracted.
Limitations
- Slower to train than an MLP with the same parameters in current implementations.
- The advantage over MLP is not general; later studies found mixed results depending on the task.
- Still a recent research architecture.
When to use it, and when not?
Good for
- Discovering symbolic relationships in scientific data.
- Approximating smooth low-dimensional functions when interpretability is wanted.
Not suited for
- High-dimensional images and text without combining it with other architectures.
- When speed is a priority.
Applications
| Domain | Use |
|---|---|
| Scientific computing | Approximating solutions of equations and discovering symbolic laws. |
| Tabular data | Interpretable models on small tabular data. |
PyTorch code
A simplified teaching code sample that runs on small synthetic data, covering: imports, model definition, data, forward pass, loss, optimizer, training loop, evaluation and inference. It is not the paper's full implementation.
The authors' official implementation is the pykan library; this code is educational and shows a KAN layer with B-spline functions without grid refinement, pruning or symbolic matching.
# Kolmogorov–Arnold Network (KAN) with B-splines — an educational implementation
# A simplified educational example in PyTorch (not the paper's full original implementation): small synthetic data,
# runs on the CPU in seconds. The numbers it prints belong to this example only.
# 1) Import the libraries
import torch
import torch.nn as nn
import torch.nn.functional as F
torch.manual_seed(0)
# 2) Define the model
class KANLayer(nn.Module):
def __init__(self, n_in, n_out, G=5, k=3, lo=-1.0, hi=1.0):
super().__init__()
h = (hi - lo) / G
self.register_buffer('t', torch.arange(-k, G + k + 1) * h + lo) # extended knots
self.k = k
self.c = nn.Parameter(torch.randn(n_out, n_in, G + k) * 0.1) # spline coefficients
self.wb = nn.Parameter(torch.ones(n_out, n_in) * 0.5) # silu base weight
def bsplines(self, x): # x: B×n_in → B×n_in×(G+k)
x = x.unsqueeze(-1); t = self.t
Bk = ((x >= t[:-1]) & (x < t[1:])).float()
for p in range(1, self.k + 1):
Bk = (x - t[:-(p + 1)]) / (t[p:-1] - t[:-(p + 1)]) * Bk[..., :-1] + \
(t[p + 1:] - x) / (t[p + 1:] - t[1:-p]) * Bk[..., 1:]
return Bk
def forward(self, x):
spline = torch.einsum('bim,oim->bo', self.bsplines(x), self.c) # Σ_i Σ_m c B(x_i)
base = F.silu(x) @ self.wb.T
return base + spline
class KAN(nn.Module):
def __init__(self, width=(2, 5, 1), G=5):
super().__init__()
self.layers = nn.ModuleList([KANLayer(a, b, G) for a, b in zip(width[:-1], width[1:])])
def forward(self, x):
for i, l in enumerate(self.layers):
x = l(x)
if i < len(self.layers) - 1:
x = torch.tanh(x) # keep the inputs inside the grid range [-1, 1] (an educational simplification)
return x
# 3) Prepare the data
# f(x, y) = exp(sin(πx) + y²) normalised, x, y ∈ [-1, 1]
def f(X):
return torch.exp(torch.sin(torch.pi * X[:, :1]) + X[:, 1:] ** 2) / 3.0
X = torch.rand(1000, 2) * 2 - 1
Y = f(X)
X_train, y_train, X_test, y_test = X[:800], Y[:800], X[800:], Y[800:]
model = KAN()
# 4) Loss function and optimizer
loss_fn = nn.MSELoss()
opt = torch.optim.Adam(model.parameters(), lr=2e-2)
# 5) Training loop with the forward pass
EPOCHS, BATCH = 60, 64
for epoch in range(EPOCHS):
model.train()
perm = torch.randperm(len(X_train))
total = 0.0
for i in range(0, len(X_train), BATCH):
idx = perm[i:i + BATCH]
xb, yb = X_train[idx], y_train[idx]
out = model(xb) # Forward Pass
loss = loss_fn(out, yb)
opt.zero_grad()
loss.backward() # Backpropagation
opt.step()
total += loss.item() * len(idx)
if epoch % max(1, EPOCHS // 4) == 0 or epoch == EPOCHS - 1:
print(f"epoch {epoch:3d} loss={total / len(X_train):.4f}")
# 6) Evaluation
model.eval()
with torch.no_grad():
xb = X_test
out = model(xb)
mse = F.mse_loss(out, y_test).item()
print(f"test MSE = {mse:.4f}")
# 7) Inference on a new sample
with torch.no_grad():
xb = X_test[:1]
out = model(xb)
print("prediction:", out.squeeze().tolist())
This code was run on 2026-11-20 in 3.8 seconds (PyTorch (CPU), Python 3 – local build machine).
Sample output:
test MSE = 0.0000
Trainable parameters in the example: model = 135.
These results are for small synthetic data and few epochs, and only prove that the code runs; do not compare them with published results.
Try it and go deeper
Related architectures
Multilayer Perceptron
Multilayer Perceptron (MLP)
A network of successive linear layers separated by nonlinear activation functions; it is the first building block on which most modern architectures are built.
Physics-Informed Neural Networks
Physics-Informed Neural Networks (PINN)
A network that approximates the solution of a partial differential equation u_θ(x,t) and is trained with a loss that combines the equation's residual (computed by automa…
Neural Operators
Neural Operators (NO)
A family of architectures that learn operators between function spaces rather than between fixed-size vectors, by stacking pointwise linear layers with integral operator…
No results match the filters.
Research ideas
- Compare KAN with an MLP at the same parameter count on PDE problems, measuring time and accuracy.
- Use KAN as an interpretable classification head on top of CNN features for medical images.
Starting suggestions, not confirmed research gaps; review the recent literature before adopting them.
Test your understanding
Where are the learnable functions in KAN?
Why? Each edge carries a univariate function, and the nodes only sum.
The other options: It is the MLP that places the fixed activation on the nodes.
What theoretical basis inspired KAN?
Why? The theorem represents a multivariate function as a composition of univariate functions and addition.
Which statement about comparing KAN with MLP is most accurate?
Why? The authors' experiments showed advantages on scientific problems, and other studies found mixed results.
References
Liu, Z., Wang, Y., Vaidya, S., Ruehle, F., Halverson, J., Soljačić, M., Hou, T. Y., & Tegmark, M. (2025). KAN: Kolmogorov–Arnold networks. International Conference on Learning Representations (ICLR). https://arxiv.org/abs/2404.19756
Kolmogorov, A. N. (1957). On the representation of continuous functions of many variables by superposition of continuous functions of one variable and addition. Doklady Akademii Nauk SSSR, 114(5), 953–956 (in Russian). English translation: American Mathematical Society Translations, Series 2, 28, 55–59 (1963). https://doi.org/10.1090/trans2/028/04
Cybenko, G. (1989). Approximation by superpositions of a sigmoidal function. Mathematics of Control, Signals and Systems, 2(4), 303–314. https://doi.org/10.1007/BF02551274
