Deep Operator Network (DeepONet)
A network that learns an operator mapping a function to a function (such as: condition → solution), through a "branch" network that encodes the input function from its values at fixed sensors and a "trunk" network that encodes the evaluation position, with the two outputs combined by a dot product.
Overview
| Name | Deep Operator Network |
|---|---|
| Year introduced | 2021 |
| Researchers | Lu Lu, Pengzhan Jin, Guofei Pang, Zhongqiang Zhang, George Em Karniadakis |
| Institution | Brown University |
| Original paper | Learning nonlinear operators via DeepONet based on the universal approximation theorem of operators DOI arXiv |
| Data type | Physical fields and functions |
| Domains | Scientific computing, Robotics, Medical image analysis |
| Computational cost | One forward pass per evaluation. |
| Data size | Hundreds to thousands of function pairs. |
Problem it solves
PINNs and similar methods solve a single problem; changing the condition or the source requires solving again. The goal is to learn the "solver" itself for instant evaluation of new cases.
Background and evolution
What came before: Multilayer Perceptron, Physics-Informed Neural Networks
Core idea
$G(u)(y)\approx\sum_{k=1}^{p}b_k(u(x_1),\dots,u(x_m))\,t_k(y)+b_0$. The branch sees the whole function through $m$ sensors, and the trunk sees a single position $y$. The dot product resembles expanding the solution in a basis of functions $t_k$ with input-dependent coefficients $b_k$.
Interactive diagram
A simplified illustrative diagram with the dimensions of a small example, not a drawing matching every detail of the original implementation. Click any layer to see its role, dimensions and mathematical operation.
Select a layer in the diagram.
Components
| Component | Role |
|---|---|
| Sensors x_1..x_m | Fixed positions for reading the function |
| Branch net | u(x_1..x_m) → b ∈ ℝᵖ |
| Trunk net | y → t ∈ ℝᵖ |
| Dot product + bias | G(u)(y) |
Equations
1. DeepONet representation
| $m$ | Number of sensors |
| $p$ | Number of basis functions |
2. The operator in the example: integration
| $u$ | Input function |
| $G(u)$ | The output function the operator learns |
| $y$ | Output evaluation point |
How it works, step by step
- Generate (input function, solution) pairs from a solver or experiments.
- Read the input function at fixed sensors.
- Choose evaluation positions.
- Train the branch and trunk with an MSE loss.
- Evaluate new functions instantly.
Strengths and limitations
Strengths
- Very fast evaluation after training.
- A theoretical basis (the Chen and Chen theorem).
- Flexibility in evaluation positions (any y).
Limitations
- Fixed sensors for the input.
- Needs training data from a solver.
- Weak generalisation outside the distribution of the training functions.
When to use it, and when not?
Good for
- Repeated evaluation of problems from the same family.
- Surrogate models.
Not suited for
- A single case only (a solver or a PINN is enough).
- Inputs on meshes that change a lot, without adaptation.
Applications
| Domain | Use |
|---|---|
| Scientific computing | Simulating fluids, heat, elasticity and inverse problems. |
| Medical image analysis | Modelling blood flow and drug diffusion in tissue. |
| Robotics | Dynamical models for control. |
PyTorch code
A simplified teaching code sample that runs on small synthetic data, covering: imports, model definition, data, forward pass, loss, optimizer, training loop, evaluation and inference. It is not the paper's full implementation.
# DeepONet for learning the integration operator
# A simplified educational example in PyTorch (not the paper's full original implementation): small synthetic data,
# runs on the CPU in seconds. The numbers it prints belong to this example only.
# 1) Import the libraries
import torch
import torch.nn as nn
import torch.nn.functional as F
torch.manual_seed(0)
# 2) Define the model
class DeepONet(nn.Module):
"""G(u)(y) ≈ Σ_k b_k(u(x_1..x_m)) · t_k(y) + b0 : a branch network for the function and a trunk network for the evaluation position."""
def __init__(self, m=50, p=40, h=64):
super().__init__()
self.branch = nn.Sequential(nn.Linear(m, h), nn.Tanh(), nn.Linear(h, h), nn.Tanh(), nn.Linear(h, p))
self.trunk = nn.Sequential(nn.Linear(1, h), nn.Tanh(), nn.Linear(h, h), nn.Tanh(), nn.Linear(h, p), nn.Tanh())
self.b0 = nn.Parameter(torch.zeros(1))
def forward(self, u_sensors, y): # u: B×m, y: B×Q×1 → B×Q
return torch.einsum('bp,bqp->bq', self.branch(u_sensors), self.trunk(y)) + self.b0
# 3) Prepare the data
# The operator to learn: integration G(u)(y) = ∫₀ʸ u(s) ds (the first example in the DeepONet paper)
# Input functions: random Fourier series; the exact integral is known analytically, so no numerical solver is needed
m, Q, K = 50, 20, 5
sensors = torch.linspace(0, 1, m)
def make(n):
a, b = torch.randn(n, K) / torch.arange(1, K + 1), torch.randn(n, K) / torch.arange(1, K + 1)
k = torch.arange(1, K + 1).float() * 2 * torch.pi
u = lambda s: (a[:, None] * torch.cos(k * s[..., None]) + b[:, None] * torch.sin(k * s[..., None])).sum(-1)
Gu = lambda y: (a[:, None] * torch.sin(k * y[..., None]) / k + b[:, None] * (1 - torch.cos(k * y[..., None])) / k).sum(-1)
y = torch.rand(n, Q)
return u(sensors.expand(n, m)), y[..., None], Gu(y)
U_train, Y_train, G_train = make(2000)
U_test, Y_test, G_test = make(300)
model = DeepONet(m)
# 4) The loss and the optimizer
opt = torch.optim.Adam(model.parameters(), lr=2e-3)
sched = torch.optim.lr_scheduler.StepLR(opt, 50, 0.4)
# 5) Training loop
EPOCHS, BATCH = 150, 100
for epoch in range(EPOCHS):
perm, tot = torch.randperm(len(U_train)), 0.0
for i in range(0, len(U_train), BATCH):
idx = perm[i:i + BATCH]
pred = model(U_train[idx], Y_train[idx]) # Forward Pass
loss = F.mse_loss(pred, G_train[idx])
opt.zero_grad(); loss.backward(); opt.step()
tot += loss.item() * len(idx)
sched.step()
if epoch % 30 == 0 or epoch == EPOCHS - 1:
print(f"epoch {epoch:3d} loss={tot / len(U_train):.2e}")
# 6) Evaluation on functions the model has not seen
with torch.no_grad():
pred = model(U_test, Y_test)
rel = (torch.norm(pred - G_test, dim=1) / torch.norm(G_test, dim=1)).mean().item()
print(f"mean relative L2 error on unseen functions = {rel:.4f}")
# 7) Inference: u(s) = cos(2πs) ⇒ the exact solution sin(2πy)/(2π); we evaluate it at y = 0.125
with torch.no_grad():
u = torch.cos(2 * torch.pi * sensors)[None]
y = torch.tensor([[[0.125]]])
print(f"G(u)(0.125): DeepONet = {model(u, y).item():.4f} ; exact = {(torch.sin(torch.tensor(2 * torch.pi * 0.125)) / (2 * torch.pi)).item():.4f}")
This code was run on 2026-11-20 in 8.7 seconds (PyTorch (CPU), Python 3 – local build machine).
Sample output:
mean relative L2 error on unseen functions = 0.1325
Trainable parameters in the example: model = 16,913.
These results are for small synthetic data and few epochs, and only prove that the code runs; do not compare them with published results.
Try it and go deeper
Related architectures
Fourier Neural Operator
Fourier Neural Operator (FNO)
A neural operator that applies a "spectral convolution" in each layer: a Fourier transform of the features, then multiplication of a limited number of low frequencies by…
Neural Operators
Neural Operators (NO)
A family of architectures that learn operators between function spaces rather than between fixed-size vectors, by stacking pointwise linear layers with integral operator…
Physics-Informed Neural Networks
Physics-Informed Neural Networks (PINN)
A network that approximates the solution of a partial differential equation u_θ(x,t) and is trained with a loss that combines the equation's residual (computed by automa…
Scientific machine learning architectures
Scientific Machine Learning Architectures (SciML)
A family of architectures that blends mechanistic models (known equations) with neural networks that learn the unknown parts, trained through differentiable solvers. Not…
No results match the filters.
Research ideas
- DeepONet as a surrogate for simulating blood flow in the coronary arteries with limited patient data.
Starting suggestions, not confirmed research gaps; review the recent literature before adopting them.
Test your understanding
What is the function of the branch network in DeepONet?
Why? The trunk encodes the position.
What is the theoretical basis of DeepONet?
Why? It proves that continuous operators can be approximated.
How are the outputs of the branch and trunk combined?
Why? $\sum_k b_kt_k+b_0$.
References
Lu, L., Jin, P., Pang, G., Zhang, Z., & Karniadakis, G. E. (2021). Learning nonlinear operators via DeepONet based on the universal approximation theorem of operators. Nature Machine Intelligence, 3(3), 218–229. https://doi.org/10.1038/s42256-021-00302-5
Chen, T., & Chen, H. (1995). Universal approximation to nonlinear operators by neural networks with arbitrary activation functions and its application to dynamical systems. IEEE Transactions on Neural Networks, 6(4), 911–917. https://doi.org/10.1109/72.392253
Kovachki, N., Li, Z., Liu, B., Azizzadenesheli, K., Bhattacharya, K., Stuart, A., & Anandkumar, A. (2023). Neural operator: Learning maps between function spaces with applications to PDEs. Journal of Machine Learning Research, 24(89), 1–97. https://arxiv.org/abs/2108.08481
Karniadakis, G. E., Kevrekidis, I. G., Lu, L., Perdikaris, P., Wang, S., & Yang, L. (2021). Physics-informed machine learning. Nature Reviews Physics, 3(6), 422–440. https://doi.org/10.1038/s42254-021-00314-5
