Gated recurrent unit (GRU)
A simplification of LSTM with only two gates (update and reset) and no separate cell state, with comparable performance on many tasks and fewer parameters.
Overview
| Name | Gated recurrent unit |
|---|---|
| Year introduced | 2014 |
| Researchers | Kyunghyun Cho, Bart van Merriënboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, Yoshua Bengio |
| Original paper | Learning phrase representations using RNN encoder–decoder for statistical machine translation DOI arXiv |
| Data type | Sequences and time series, Text |
| Domains | Time series forecasting, Audio and speech, Natural language processing |
Problem it solves
LSTM is effective but has many parameters; researchers wanted a simpler unit that keeps the benefit of gates.
Background and evolution
What came before: Long short-term memory, Recurrent neural network
Core idea
The update gate $z_t$ decides how much of the old state is kept versus the new candidate state (linear interpolation). The reset gate $r_t$ decides how much of the previous state is used when computing the candidate.
Interactive diagram
A simplified illustrative diagram with the dimensions of a small example, not a drawing matching every detail of the original implementation. Click any layer to see its role, dimensions and mathematical operation.
Select a layer in the diagram.
Components
| Component | Role |
|---|---|
| Update gate z_t | Balancing old and new |
| Reset gate r_t | Effect of the past on the candidate |
| Candidate ĥ_t | Proposed state |
| Hidden state h_t | Linear interpolation |
Equations
1. GRU equations (PyTorch form)
| $z_t$ | Update gate |
| $r_t$ | Reset gate |
How it works, step by step
- The update and reset gates are computed.
- The candidate state is computed.
- Interpolate between the old state and the candidate.
- Repeat for every step.
Strengths and limitations
Strengths
- Fewer parameters than LSTM (three sets instead of four).
- Comparable performance on many tasks.
Limitations
- No separation between memory and output as in LSTM.
- Does not consistently beat LSTM; the choice is empirical.
When to use it, and when not?
Good for
- When resources are limited and memory is needed.
Not suited for
- Very long contexts that require parallelism.
Applications
| Domain | Use |
|---|---|
| Time series forecasting | Forecasting demand, energy and prices. |
| Natural language processing | Language modelling and speech recognition (historically). |
| Medical image analysis | Analysing biosignals such as ECG and EEG. |
| Robotics | Sequential control and motion prediction. |
PyTorch code
A simplified teaching code sample that runs on small synthetic data, covering: imports, model definition, data, forward pass, loss, optimizer, training loop, evaluation and inference. It is not the paper's full implementation.
# GRU — next-value prediction
# A simplified educational example in PyTorch (not the paper's full original implementation): small synthetic data,
# runs on the CPU in seconds. The numbers it prints belong to this example only.
# 1) Import the libraries
import torch
import torch.nn as nn
import torch.nn.functional as F
torch.manual_seed(0)
# 2) Define the model
class GRUForecaster(nn.Module):
def __init__(self, d_in=1, h=32):
super().__init__()
self.gru = nn.GRU(d_in, h, batch_first=True)
self.out = nn.Linear(h, 1)
def forward(self, x):
H, _ = self.gru(x)
return self.out(H[:, -1])
# 3) Prepare the data
# Predicting the next value of a sum of two waves with random phases and frequencies
def make(n, T=30):
w1, w2 = torch.rand(n, 1) * 0.3 + 0.1, torch.rand(n, 1) * 0.1 + 0.02
t = torch.rand(n, 1) * 50 + torch.arange(T + 1).float()
s = torch.sin(w1 * t) + 0.5 * torch.sin(w2 * t) + 0.03 * torch.randn(n, T + 1)
return s[:, :T, None], s[:, T:]
X_train, y_train = make(800)
X_test, y_test = make(200)
model = GRUForecaster()
# 4) Loss function and optimizer
loss_fn = nn.MSELoss()
opt = torch.optim.Adam(model.parameters(), lr=5e-3)
# 5) Training loop with the forward pass
EPOCHS, BATCH = 15, 32
for epoch in range(EPOCHS):
model.train()
perm = torch.randperm(len(X_train))
total = 0.0
for i in range(0, len(X_train), BATCH):
idx = perm[i:i + BATCH]
xb, yb = X_train[idx], y_train[idx]
out = model(xb) # Forward Pass
loss = loss_fn(out, yb)
opt.zero_grad()
loss.backward() # Backpropagation
opt.step()
total += loss.item() * len(idx)
if epoch % max(1, EPOCHS // 4) == 0 or epoch == EPOCHS - 1:
print(f"epoch {epoch:3d} loss={total / len(X_train):.4f}")
# 6) Evaluation
model.eval()
with torch.no_grad():
xb = X_test
out = model(xb)
mse = F.mse_loss(out, y_test).item()
print(f"test MSE = {mse:.4f}")
# 7) Inference on a new sample
with torch.no_grad():
xb = X_test[:1]
out = model(xb)
print("prediction:", out.squeeze().tolist())
This code was run on 2026-11-20 in 4.7 seconds (PyTorch (CPU), Python 3 – local build machine).
Sample output:
test MSE = 0.0047
Trainable parameters in the example: model = 3,393.
These results are for small synthetic data and few epochs, and only prove that the code runs; do not compare them with published results.
Try it and go deeper
- Neural Architecture Designer
- AI Model Atlas Another section of the site
- Time Series Lab Another section of the site
Related architectures
Long short-term memory
Long Short-Term Memory (LSTM)
A recurrent network with a memory cell and gates (forget, input, output) that control what is kept and what is erased, so it learns long-term dependencies far better tha…
Recurrent neural network
Recurrent Neural Network (RNN)
A network that processes a sequence element by element and carries a hidden state that passes information from previous steps, using the same weights at every step.
Hybrid CNN–GRU Networks
CNN-GRU
One-dimensional convolutions capture local patterns in the signal and shorten it, then a GRU (often bidirectional) models relationships over time with fewer parameters t…
Temporal Graph Networks
Temporal Graph Networks (TGN)
A framework for learning representations on dynamic graphs represented as a stream of timed events, using a memory for each node that is updated by events and an embeddi…
No results match the filters.
Research ideas
- A systematic comparison of GRU and LSTM on Arabic EEG data with equal hyperparameter tuning.
Starting suggestions, not confirmed research gaps; review the recent literature before adopting them.
Test your understanding
How many gates does a GRU have?
Why? LSTM is the one with three gates and a cell state.
In the PyTorch form, if $z_t=1$, what is $h_t$?
Why? $h_t=(1-1)\tilde h+1\cdot h_{t-1}$.
Which claim about GRU vs LSTM is most accurate?
Why? The comparisons by Chung et al. concluded that performance is similar.
References
Cho, K., van Merriënboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014). Learning phrase representations using RNN encoder–decoder for statistical machine translation. Proceedings of EMNLP 2014, 1724–1734. https://doi.org/10.3115/v1/D14-1179
Chung, J., Gulcehre, C., Cho, K., & Bengio, Y. (2014). Empirical evaluation of gated recurrent neural networks on sequence modeling. NIPS 2014 Workshop on Deep Learning. https://arxiv.org/abs/1412.3555
Hochreiter, S., & Schmidhuber, J. (1997). Long short-term memory. Neural Computation, 9(8), 1735–1780. https://doi.org/10.1162/neco.1997.9.8.1735
