AdvancedPhysics-Informed Neural Networks

Physics-informed deep learning (PIML)

A general approach to embedding physical knowledge in deep learning in three ways: through data (observational bias), through the architecture (inductive bias, such as Hamiltonian networks and equivariance), or through the loss function (learning bias, such as PINN).

Overview

NamePhysics-informed deep learning
Year introduced2021
ResearchersReference review: George Em Karniadakis, Ioannis G. Kevrekidis, Lu Lu, Paris Perdikaris, Sifan Wang, Liu Yang; architecture example: Greydanus et al. (HNN 2019)
Original paperPhysics-informed machine learning DOI
Data typePhysical fields and functions, Sequences and time series
DomainsScientific computing, Robotics, Medical image analysis
Computational costLike the base network plus the cost of extra gradients.
Data sizeSmall to medium.

Problem it solves

Purely data-driven models may violate conservation laws (energy, symmetry), so their long-horizon predictions degrade and they generalize poorly outside the training data.

Background and evolution

What came before: Multilayer Perceptron, Physics-Informed Neural Networks, Neural ordinary differential equations

  1. 2019

    HNN: learns the Hamiltonian and derives the dynamics from it to conserve energy. [ref]

  2. 2020

    LNN: learns the Lagrangian in generalized coordinates. [ref]

  3. 2021

    The Nature Reviews Physics review classified ways of embedding physics into three. [ref]

Core idea

Instead of predicting the derivatives directly, the network learns a scalar $H_\theta(q,p)$ and the dynamics are computed from Hamilton's equations. Because the resulting flow is Hamiltonian, $H_\theta$ is conserved along the trajectory (with an accurate integrator), so energy does not drift. This is an example of "inductive bias"; PINN is an example of "learning bias".

Example: A robot-arm motion model that conserves energy in long simulations instead of gaining spurious energy over time.

Interactive diagram

A simplified illustrative diagram with the dimensions of a small example, not a drawing matching every detail of the original implementation. Click any layer to see its role, dimensions and mathematical operation.

Select a layer in the diagram.

Components

ComponentRole
Observational biasData that respects symmetries, or data augmentation
Inductive bias (architecture)HNN, LNN, equivariant networks
Learning bias (loss)The remaining equations as in PINN
Differentiable simulatorOptional

Equations

1. Hamilton's equations in HNN

$$\frac{dq}{dt}=\frac{\partial H_\theta}{\partial p},\qquad\frac{dp}{dt}=-\frac{\partial H_\theta}{\partial q}$$

Symbols
$q,p$Position and momentum
$H_\theta$Learned Hamiltonian
Dimensions
Vector of length 2d
Numerical example
$H=(q^2+p^2)/2$ at $(q,p)=(1,0)$: $\dot q=0$ and $\dot p=-1$.
Relation to the architecture
The dynamics are derived from a conserved quantity.

2. Energy conservation

$$\frac{dH}{dt}=\frac{\partial H}{\partial q}\dot q+\frac{\partial H}{\partial p}\dot p=\frac{\partial H}{\partial q}\frac{\partial H}{\partial p}-\frac{\partial H}{\partial p}\frac{\partial H}{\partial q}=0$$

Symbols
$H$Hamiltonian function (energy)
$q,p$Position and momentum
$\dot q,\dot p$Their time derivatives
Dimensions
A scalar
Numerical example
In the example: after simulating up to $t=100$, the HNN energy stayed near $0.5$ while it almost doubled for the plain network.
Relation to the architecture
The reason for stability in long-horizon prediction.

3. HNN loss

$$\mathcal{L}=\left\lVert\frac{\partial H_\theta}{\partial p}-\dot q\right\rVert^2+\left\lVert\frac{\partial H_\theta}{\partial q}+\dot p\right\rVert^2$$

Symbols
$H_\theta$Energy predicted by the network
$\dot q,\dot p$Derivatives measured from the data
Dimensions
A scalar
Numerical example
If $\partial H/\partial p=0.9$ and $\dot q=1$: the first term is $0.01$.
Relation to the architecture
The same derivative-matching objective, but through a constrained structure.

How it works, step by step

  1. Identify the known quantities or symmetries.
  2. Choose the embedding route: data, architecture, loss.
  3. Design the model (e.g. HNN).
  4. Train on the data.
  5. Test conservation and generalization in a long simulation.

Strengths and limitations

Strengths

  • Better generalization with less data.
  • Respects conservation laws.
  • More stable long-horizon predictions.

Limitations

  • Needs correct physical knowledge; a wrong assumption hurts.
  • Dissipative systems do not suit pure HNNs.
  • Extra differentiation cost.

When to use it, and when not?

Good for

  • Physical systems with known conservation laws or symmetries.

Not suited for

  • Systems with no known physics, or strongly dissipative ones without modification.

Applications

DomainUse
Scientific computingSimulating fluids, heat, elasticity and inverse problems.
Medical image analysisModelling blood flow and drug diffusion in tissue.
RoboticsDynamical models for control.

PyTorch code

A simplified teaching code sample that runs on small synthetic data, covering: imports, model definition, data, forward pass, loss, optimizer, training loop, evaluation and inference. It is not the paper's full implementation.

Python (PyTorch)
# Hamiltonian Neural Network vs a plain network: energy drift
# A simplified educational example in PyTorch (not the paper's full original implementation): small synthetic data,
# runs on the CPU in seconds. The numbers it prints belong to this example only.
# 1) Import the libraries
import torch
import torch.nn as nn
import torch.nn.functional as F

torch.manual_seed(0)

# 2) Define the model
def mlp(i, o, h=64):
    return nn.Sequential(nn.Linear(i, h), nn.Tanh(), nn.Linear(h, h), nn.Tanh(), nn.Linear(h, o))

class HNN(nn.Module):
    """Hamiltonian network: learns the scalar H_θ(q, p), and the dynamics are derived from it: dq/dt = ∂H/∂p , dp/dt = −∂H/∂q."""
    def __init__(self):
        super().__init__(); self.H = mlp(2, 1)
    def forward(self, x):                       # x = (q, p)
        with torch.enable_grad():
            x = x if x.requires_grad else x.requires_grad_(True)
            dH = torch.autograd.grad(self.H(x).sum(), x, create_graph=True)[0]
        return torch.stack([dH[:, 1], -dH[:, 0]], 1)

class Baseline(nn.Module):                      # plain network that predicts derivatives directly, with no physical structure
    def __init__(self):
        super().__init__(); self.f = mlp(2, 2)
    def forward(self, x):
        return self.f(x)

# 3) Prepare the data
# Ideal spring (k = m = 1): H = (q² + p²)/2 , energy is conserved. Data: states and their derivatives with measurement noise.
def make(n):
    r = torch.rand(n) * 0.9 + 0.1; th = torch.rand(n) * 2 * torch.pi
    x = torch.stack([r * th.cos(), r * th.sin()], 1)
    dx = torch.stack([x[:, 1], -x[:, 0]], 1) + 0.05 * torch.randn(n, 2)
    return x, dx
X_train, dX_train = make(1500)
X_test, dX_test = make(300)
models = {'baseline': Baseline(), 'hnn': HNN()}

# 4) and 5) The same loss for both models: match the measured time derivatives, then train
for name, model in models.items():
    opt = torch.optim.Adam(model.parameters(), lr=1e-3)
    for epoch in range(1500):
        loss = F.mse_loss(model(X_train), dX_train)                         # Forward Pass
        opt.zero_grad(); loss.backward(); opt.step()
    print(f"{name:8s} train loss = {loss.item():.5f}")

# 6) Evaluation: derivative error, then energy drift in a long simulation (RK4, 2000 steps)
def rk4(f, x, dt):
    a = f(x); b = f(x + dt / 2 * a); c = f(x + dt / 2 * b); d = f(x + dt * c)
    return (x + dt / 6 * (a + 2 * b + 2 * c + d)).detach()
energy = lambda x: 0.5 * (x ** 2).sum(1)
for name, model in models.items():
    test_mse = F.mse_loss(model(X_test).detach(), dX_test).item()
    x = torch.tensor([[1.0, 0.0]])
    for _ in range(2000):
        x = rk4(model, x, 0.05)
    print(f"{name:8s} test derivative MSE = {test_mse:.5f} ; energy after t=100: {energy(x).item():.3f} (true 0.500)")

# 7) Inference: value of the learned Hamiltonian (defined up to an additive constant)
with torch.no_grad():
    H = models['hnn'].H
    print("learned H(1,0) − H(0,0) =", round((H(torch.tensor([[1.0, 0.0]])) - H(torch.zeros(1, 2))).item(), 3), "(true 0.5)")

This code was run on 2026-11-20 in 8.9 seconds (PyTorch (CPU), Python 3 – local build machine).

Sample output:

baseline test derivative MSE = 0.00255 ; energy after t=100: 1.048 (true 0.500)
hnn      test derivative MSE = 0.00254 ; energy after t=100: 0.502 (true 0.500)

Trainable parameters in the example: model = 4,417, H = 4,417.

These results are for small synthetic data and few epochs, and only prove that the code runs; do not compare them with published results.

Try it and go deeper

Related architectures

Intermediate2019

Physics-Informed Neural Networks

Physics-Informed Neural Networks (PINN)

A network that approximates the solution of a partial differential equation u_θ(x,t) and is trained with a loss that combines the equation's residual (computed by automa…

Physics-Informed Neural Networks

Advanced2020

Scientific machine learning architectures

Scientific Machine Learning Architectures (SciML)

A family of architectures that blends mechanistic models (known equations) with neural networks that learn the unknown parts, trained through differentiable solvers. Not…

Physics-Informed Neural NetworksHybrid Architectures

Research2016

Equivariant neural networks

Equivariant Neural Networks (ENN)

Networks built so that their output transforms in a specific way when the input is transformed by a symmetry (rotation, translation, reflection, permutation), so they do…

Specialized Architectures

Advanced2018

Neural ordinary differential equations

Neural Ordinary Differential Equations (Neural ODE)

The hidden state is defined by a continuous differential equation whose derivative is given by a neural network, and the output is computed with a numerical solver, so t…

Temporal and Sequential NetworksPhysics-Informed Neural Networks

Advanced2021

Liquid Neural Networks

Liquid Neural Networks (LNN)

Continuous-time recurrent networks in which each neuron's time constant changes with the input (Liquid Time-Constant), so their dynamics adapt to the data with a small n…

Temporal and Sequential NetworksAdvanced Neural Architectures

Research ideas

  • Hamiltonian networks with learned dissipation for modelling prosthetic limb motion.

Starting suggestions, not confirmed research gaps; review the recent literature before adopting them.

Test your understanding

  1. Concept

    What are the three ways of embedding physics according to the review by Karniadakis et al.?

  2. Concept

    Why does an HNN conserve the learned energy?

  3. Mathematical

    With $H=(q^2+p^2)/2$ at $(q,p)=(0,2)$, what is $\dot q$?

References

  1. Karniadakis, G. E., Kevrekidis, I. G., Lu, L., Perdikaris, P., Wang, S., & Yang, L. (2021). Physics-informed machine learning. Nature Reviews Physics, 3(6), 422–440. https://doi.org/10.1038/s42254-021-00314-5

    DOIAutomatically verified via Crossref – 2026-11-20

  2. Greydanus, S., Dzamba, M., & Yosinski, J. (2019). Hamiltonian neural networks. Advances in Neural Information Processing Systems (NeurIPS) 32. https://arxiv.org/abs/1906.01563

    arXivAutomatically verified via OpenAlex/arXiv – 2026-11-20

  3. Cranmer, M., Greydanus, S., Hoyer, S., Battaglia, P., Spergel, D., & Ho, S. (2020). Lagrangian neural networks. ICLR 2020 Workshop on Integration of Deep Neural Models and Differential Equations. https://arxiv.org/abs/2003.04630

    arXivAutomatically verified via OpenAlex/arXiv – 2026-11-20

  4. Raissi, M., Perdikaris, P., & Karniadakis, G. E. (2019). Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational Physics, 378, 686–707. https://doi.org/10.1016/j.jcp.2018.10.045

    DOIAutomatically verified via Crossref – 2026-11-20