Physics-informed deep learning (PIML)
A general approach to embedding physical knowledge in deep learning in three ways: through data (observational bias), through the architecture (inductive bias, such as Hamiltonian networks and equivariance), or through the loss function (learning bias, such as PINN).
Overview
| Name | Physics-informed deep learning |
|---|---|
| Year introduced | 2021 |
| Researchers | Reference review: George Em Karniadakis, Ioannis G. Kevrekidis, Lu Lu, Paris Perdikaris, Sifan Wang, Liu Yang; architecture example: Greydanus et al. (HNN 2019) |
| Original paper | Physics-informed machine learning DOI |
| Data type | Physical fields and functions, Sequences and time series |
| Domains | Scientific computing, Robotics, Medical image analysis |
| Computational cost | Like the base network plus the cost of extra gradients. |
| Data size | Small to medium. |
Problem it solves
Purely data-driven models may violate conservation laws (energy, symmetry), so their long-horizon predictions degrade and they generalize poorly outside the training data.
Background and evolution
What came before: Multilayer Perceptron, Physics-Informed Neural Networks, Neural ordinary differential equations
Core idea
Instead of predicting the derivatives directly, the network learns a scalar $H_\theta(q,p)$ and the dynamics are computed from Hamilton's equations. Because the resulting flow is Hamiltonian, $H_\theta$ is conserved along the trajectory (with an accurate integrator), so energy does not drift. This is an example of "inductive bias"; PINN is an example of "learning bias".
Interactive diagram
A simplified illustrative diagram with the dimensions of a small example, not a drawing matching every detail of the original implementation. Click any layer to see its role, dimensions and mathematical operation.
Select a layer in the diagram.
Components
| Component | Role |
|---|---|
| Observational bias | Data that respects symmetries, or data augmentation |
| Inductive bias (architecture) | HNN, LNN, equivariant networks |
| Learning bias (loss) | The remaining equations as in PINN |
| Differentiable simulator | Optional |
Equations
1. Hamilton's equations in HNN
| $q,p$ | Position and momentum |
| $H_\theta$ | Learned Hamiltonian |
2. Energy conservation
| $H$ | Hamiltonian function (energy) |
| $q,p$ | Position and momentum |
| $\dot q,\dot p$ | Their time derivatives |
3. HNN loss
| $H_\theta$ | Energy predicted by the network |
| $\dot q,\dot p$ | Derivatives measured from the data |
How it works, step by step
- Identify the known quantities or symmetries.
- Choose the embedding route: data, architecture, loss.
- Design the model (e.g. HNN).
- Train on the data.
- Test conservation and generalization in a long simulation.
Strengths and limitations
Strengths
- Better generalization with less data.
- Respects conservation laws.
- More stable long-horizon predictions.
Limitations
- Needs correct physical knowledge; a wrong assumption hurts.
- Dissipative systems do not suit pure HNNs.
- Extra differentiation cost.
When to use it, and when not?
Good for
- Physical systems with known conservation laws or symmetries.
Not suited for
- Systems with no known physics, or strongly dissipative ones without modification.
Applications
| Domain | Use |
|---|---|
| Scientific computing | Simulating fluids, heat, elasticity and inverse problems. |
| Medical image analysis | Modelling blood flow and drug diffusion in tissue. |
| Robotics | Dynamical models for control. |
PyTorch code
A simplified teaching code sample that runs on small synthetic data, covering: imports, model definition, data, forward pass, loss, optimizer, training loop, evaluation and inference. It is not the paper's full implementation.
# Hamiltonian Neural Network vs a plain network: energy drift
# A simplified educational example in PyTorch (not the paper's full original implementation): small synthetic data,
# runs on the CPU in seconds. The numbers it prints belong to this example only.
# 1) Import the libraries
import torch
import torch.nn as nn
import torch.nn.functional as F
torch.manual_seed(0)
# 2) Define the model
def mlp(i, o, h=64):
return nn.Sequential(nn.Linear(i, h), nn.Tanh(), nn.Linear(h, h), nn.Tanh(), nn.Linear(h, o))
class HNN(nn.Module):
"""Hamiltonian network: learns the scalar H_θ(q, p), and the dynamics are derived from it: dq/dt = ∂H/∂p , dp/dt = −∂H/∂q."""
def __init__(self):
super().__init__(); self.H = mlp(2, 1)
def forward(self, x): # x = (q, p)
with torch.enable_grad():
x = x if x.requires_grad else x.requires_grad_(True)
dH = torch.autograd.grad(self.H(x).sum(), x, create_graph=True)[0]
return torch.stack([dH[:, 1], -dH[:, 0]], 1)
class Baseline(nn.Module): # plain network that predicts derivatives directly, with no physical structure
def __init__(self):
super().__init__(); self.f = mlp(2, 2)
def forward(self, x):
return self.f(x)
# 3) Prepare the data
# Ideal spring (k = m = 1): H = (q² + p²)/2 , energy is conserved. Data: states and their derivatives with measurement noise.
def make(n):
r = torch.rand(n) * 0.9 + 0.1; th = torch.rand(n) * 2 * torch.pi
x = torch.stack([r * th.cos(), r * th.sin()], 1)
dx = torch.stack([x[:, 1], -x[:, 0]], 1) + 0.05 * torch.randn(n, 2)
return x, dx
X_train, dX_train = make(1500)
X_test, dX_test = make(300)
models = {'baseline': Baseline(), 'hnn': HNN()}
# 4) and 5) The same loss for both models: match the measured time derivatives, then train
for name, model in models.items():
opt = torch.optim.Adam(model.parameters(), lr=1e-3)
for epoch in range(1500):
loss = F.mse_loss(model(X_train), dX_train) # Forward Pass
opt.zero_grad(); loss.backward(); opt.step()
print(f"{name:8s} train loss = {loss.item():.5f}")
# 6) Evaluation: derivative error, then energy drift in a long simulation (RK4, 2000 steps)
def rk4(f, x, dt):
a = f(x); b = f(x + dt / 2 * a); c = f(x + dt / 2 * b); d = f(x + dt * c)
return (x + dt / 6 * (a + 2 * b + 2 * c + d)).detach()
energy = lambda x: 0.5 * (x ** 2).sum(1)
for name, model in models.items():
test_mse = F.mse_loss(model(X_test).detach(), dX_test).item()
x = torch.tensor([[1.0, 0.0]])
for _ in range(2000):
x = rk4(model, x, 0.05)
print(f"{name:8s} test derivative MSE = {test_mse:.5f} ; energy after t=100: {energy(x).item():.3f} (true 0.500)")
# 7) Inference: value of the learned Hamiltonian (defined up to an additive constant)
with torch.no_grad():
H = models['hnn'].H
print("learned H(1,0) − H(0,0) =", round((H(torch.tensor([[1.0, 0.0]])) - H(torch.zeros(1, 2))).item(), 3), "(true 0.5)")
This code was run on 2026-11-20 in 8.9 seconds (PyTorch (CPU), Python 3 – local build machine).
Sample output:
baseline test derivative MSE = 0.00255 ; energy after t=100: 1.048 (true 0.500) hnn test derivative MSE = 0.00254 ; energy after t=100: 0.502 (true 0.500)
Trainable parameters in the example: model = 4,417, H = 4,417.
These results are for small synthetic data and few epochs, and only prove that the code runs; do not compare them with published results.
Try it and go deeper
Related architectures
Physics-Informed Neural Networks
Physics-Informed Neural Networks (PINN)
A network that approximates the solution of a partial differential equation u_θ(x,t) and is trained with a loss that combines the equation's residual (computed by automa…
Scientific machine learning architectures
Scientific Machine Learning Architectures (SciML)
A family of architectures that blends mechanistic models (known equations) with neural networks that learn the unknown parts, trained through differentiable solvers. Not…
Equivariant neural networks
Equivariant Neural Networks (ENN)
Networks built so that their output transforms in a specific way when the input is transformed by a symmetry (rotation, translation, reflection, permutation), so they do…
Neural ordinary differential equations
Neural Ordinary Differential Equations (Neural ODE)
The hidden state is defined by a continuous differential equation whose derivative is given by a neural network, and the output is computed with a numerical solver, so t…
Liquid Neural Networks
Liquid Neural Networks (LNN)
Continuous-time recurrent networks in which each neuron's time constant changes with the input (Liquid Time-Constant), so their dynamics adapt to the data with a small n…
No results match the filters.
Research ideas
- Hamiltonian networks with learned dissipation for modelling prosthetic limb motion.
Starting suggestions, not confirmed research gaps; review the recent literature before adopting them.
Test your understanding
What are the three ways of embedding physics according to the review by Karniadakis et al.?
Why? Observational, inductive and learning bias.
Why does an HNN conserve the learned energy?
Why? Differentiation gives exactly zero.
With $H=(q^2+p^2)/2$ at $(q,p)=(0,2)$, what is $\dot q$?
Why? $\partial H/\partial p=p=2$.
References
Karniadakis, G. E., Kevrekidis, I. G., Lu, L., Perdikaris, P., Wang, S., & Yang, L. (2021). Physics-informed machine learning. Nature Reviews Physics, 3(6), 422–440. https://doi.org/10.1038/s42254-021-00314-5
Greydanus, S., Dzamba, M., & Yosinski, J. (2019). Hamiltonian neural networks. Advances in Neural Information Processing Systems (NeurIPS) 32. https://arxiv.org/abs/1906.01563
Cranmer, M., Greydanus, S., Hoyer, S., Battaglia, P., Spergel, D., & Ho, S. (2020). Lagrangian neural networks. ICLR 2020 Workshop on Integration of Deep Neural Models and Differential Equations. https://arxiv.org/abs/2003.04630
Raissi, M., Perdikaris, P., & Karniadakis, G. E. (2019). Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational Physics, 378, 686–707. https://doi.org/10.1016/j.jcp.2018.10.045
