Temporal Graph Networks (TGN)
A framework for learning representations on dynamic graphs represented as a stream of timed events, using a memory for each node that is updated by events and an embedding module that aggregates temporal neighbours.
Overview
| Name | Temporal Graph Networks |
|---|---|
| Year introduced | 2020 |
| Researchers | Emanuele Rossi, Ben Chamberlain, Fabrizio Frasca, Davide Eynard, Federico Monti, Michael Bronstein |
| Institution | |
| Original paper | Temporal graph networks for deep learning on dynamic graphs arXiv |
| Data type | Graphs, Sequences and time series |
| Domains | Recommender systems, Fraud detection, Social network analysis, Learning on graph data |
Problem it solves
Many graphs change continuously (interactions, transactions, messages), and static models ignore the timing and order of events.
Background and evolution
What came before: GraphSAGE, Gated recurrent unit, JODIE/DyRep/TGAT
- 2020
Rossi and colleagues introduced TGN as a general framework that includes earlier models as special cases. [ref]
Core idea
Each node has a memory vector $s_i(t)$ that summarises its history. When an event occurs between $i$ and $j$ at time $t$, a message is built from their two memories, the time gap and the event features, and the memory is updated with a GRU. At prediction time the node embedding is computed from its memory and its temporal neighbours with attention or aggregation.
Interactive diagram
A simplified illustrative diagram with the dimensions of a small example, not a drawing matching every detail of the original implementation. Click any layer to see its role, dimensions and mathematical operation.
Select a layer in the diagram.
Components
| Component | Role |
|---|---|
| Event stream | Events $(i,j,t,e_{ij})$ |
| Message function | From the two memories, the time and the event |
| Time encoding | A periodic encoding of $\Delta t$ |
| Memory updater | A GRU for each node |
| Embedding module | Attention over temporal neighbours |
| Decoder | Predicting future links |
Equations
1. Memory update
| $s_i(t^-)$ | Memory before the event |
| $\phi$ | Time encoding |
| $t_i^-$ | Time of the node's last event |
2. Time encoding
| $\omega,b$ | Learned vectors |
How it works, step by step
- Events are ordered in time and processed in batches.
- Embeddings are computed from the current memory to predict the batch's events.
- The loss is computed against negative samples.
- Messages are built from the batch's events.
- The memory is updated after the computation (to avoid information leakage).
Strengths and limitations
Strengths
- It captures the dynamics of graphs and the timing of events.
- A general framework that includes earlier models.
Limitations
- The update order is sensitive; a mistake in it leaks data from the future.
- Training is relatively sequential and harder to parallelise.
When to use it, and when not?
Good for
- Predicting upcoming interactions.
- Fraud detection in transaction streams.
Not suited for
- Static graphs or a few widely spaced snapshots.
Applications
| Domain | Use |
|---|---|
| Recommender systems | Predicting the next interaction between a user and an item. |
| Fraud detection | Detecting anomalous behaviour in a transaction stream. |
| Social network analysis | Predicting follows and messages. |
PyTorch code
A simplified teaching code sample that runs on small synthetic data, covering: imports, model definition, data, forward pass, loss, optimizer, training loop, evaluation and inference. It is not the paper's full implementation.
# A simplified Temporal Graph Network (TGN) for link prediction
# A simplified educational example in PyTorch (not the paper's full original implementation): small synthetic data,
# runs on the CPU in seconds. The numbers it prints belong to this example only.
# 1) Import the libraries
import torch
import torch.nn as nn
import torch.nn.functional as F
torch.manual_seed(0)
# 2) Define the model
class TimeEnc(nn.Module):
def __init__(self, d):
super().__init__(); self.lin = nn.Linear(1, d)
def forward(self, dt):
return torch.cos(self.lin(dt[:, None])) # φ(Δt)
class TGN(nn.Module):
def __init__(self, N, d=16, dt_dim=8):
super().__init__()
self.N, self.d = N, d
self.time = TimeEnc(dt_dim)
self.msg = nn.Linear(2 * d + dt_dim, d)
self.gru = nn.GRUCell(d, d)
self.emb = nn.Embedding(N, d) # fixed node features
self.dec = nn.Sequential(nn.Linear(2 * d, d), nn.ReLU(), nn.Linear(d, 1))
self.reset()
def reset(self):
self.mem = torch.zeros(self.N, self.d); self.last = torch.zeros(self.N)
def z(self, nodes):
return self.mem[nodes] + self.emb(nodes) # embedding = memory + feature
def score(self, src, dst):
return self.dec(torch.cat([self.z(src), self.z(dst)], 1)).squeeze(1)
def update(self, src, dst, t):
for a, b in ((src, dst), (dst, src)):
m = self.msg(torch.cat([self.mem[a], self.mem[b], self.time(t - self.last[a])], 1))
self.mem = self.mem.clone(); self.mem[a] = self.gru(torch.relu(m), self.mem[a])
self.last = self.last.clone(); self.last[a] = t
# 3) Prepare the data
# A synthetic event stream: 40 users in 4 groups, interacting mostly within their group
N = 40
group = torch.arange(N) % 4
src = torch.randint(0, N, (1200,))
same = torch.rand(1200) < 0.85
dst = torch.where(same, (src + 4 * torch.randint(1, 10, (1200,))) % N, torch.randint(0, N, (1200,)))
t = torch.arange(1200).float()
split = 1000
model = TGN(N)
# 4) The loss: BCE for a real event against a random negative event
loss_fn = nn.BCEWithLogitsLoss()
opt = torch.optim.Adam(model.parameters(), lr=1e-2)
# 5) Training loop: we predict the batch first, then update the memory (preventing leakage from the future)
for epoch in range(4):
model.reset(); total = 0.0
for i in range(0, split, 50):
s, d_, tt = src[i:i + 50], dst[i:i + 50], t[i:i + 50]
neg = torch.randint(0, N, (len(s),))
logits = torch.cat([model.score(s, d_), model.score(s, neg)])
labels = torch.cat([torch.ones(len(s)), torch.zeros(len(s))])
loss = loss_fn(logits, labels)
opt.zero_grad(); loss.backward(); opt.step(); total += loss.item()
with torch.no_grad():
model.update(s, d_, tt)
model.mem, model.last = model.mem.detach(), model.last.detach()
print(f"epoch {epoch} loss={total / (split // 50):.4f}")
# 6) Evaluation on the later events in time
with torch.no_grad():
s, d_ = src[split:], dst[split:]
pos, neg = model.score(s, d_), model.score(s, torch.randint(0, N, (len(s),)))
acc = ((pos > neg).float().mean()).item()
print(f"test accuracy = {acc:.3f} (share of real events scored higher than the negative)")
# 7) Inference: probability that user 0 interacts with 4 (same group) and with 1 (another group)
with torch.no_grad():
print("p(0→4) =", torch.sigmoid(model.score(torch.tensor([0]), torch.tensor([4]))).item(),
" p(0→1) =", torch.sigmoid(model.score(torch.tensor([0]), torch.tensor([1]))).item())
This code was run on 2026-11-20 in 2.9 seconds (PyTorch (CPU), Python 3 – local build machine).
Sample output:
test accuracy = 0.705 (share of real events scored higher than the negative)
Trainable parameters in the example: model = 3,489.
These results are for small synthetic data and few epochs, and only prove that the code runs; do not compare them with published results.
Try it and go deeper
- Graph Network Explorer
- Graph Neural Networks Lab Another section of the site
- Time Series Lab Another section of the site
Related architectures
GraphSAGE
GraphSAGE
An inductive framework that learns aggregation functions over a sampled neighbourhood, so it can compute representations for new nodes the network did not see during tra…
Gated recurrent unit
Gated Recurrent Unit (GRU)
A simplification of LSTM with only two gates (update and reset) and no separate cell state, with comparable performance on many tasks and fewer parameters.
Graph Attention Network
Graph Attention Network (GAT)
It replaces GCN's fixed weights with learned attention coefficients for each edge, so each node decides how important each neighbour is, with multiple attention heads.
No results match the filters.
Research ideas
- Apply TGN to fraud detection in transactions with a strict temporal split and evaluation by AUC-PR.
Starting suggestions, not confirmed research gaps; review the recent literature before adopting them.
Test your understanding
What is the function of the memory in TGN?
Why? A vector $s_i(t)$ updated with each event.
Why is the memory updated after predicting the batch and not before?
Why? Updating before predicting leaks the future into the model.
What does $\phi(\Delta t)$ encode?
Why? It helps the model know about recent activity and gaps.
References
Rossi, E., Chamberlain, B., Frasca, F., Eynard, D., Monti, F., & Bronstein, M. (2020). Temporal graph networks for deep learning on dynamic graphs. ICML 2020 Workshop on Graph Representation Learning and Beyond. https://arxiv.org/abs/2006.10637
Hamilton, W. L., Ying, R., & Leskovec, J. (2017). Inductive representation learning on large graphs. Advances in Neural Information Processing Systems (NeurIPS) 30. https://arxiv.org/abs/1706.02216
Cho, K., van Merriënboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014). Learning phrase representations using RNN encoder–decoder for statistical machine translation. Proceedings of EMNLP 2014, 1724–1734. https://doi.org/10.3115/v1/D14-1179
