Hybrid CNN–GRU Networks
CNN-GRU
One-dimensional convolutions capture local patterns in the signal and shorten it, then a GRU (often bidirectional) models relationships over time with fewer parameters than an LSTM.
Overview
| Name | Hybrid CNN–GRU Networks – CNN-GRU |
|---|---|
| Year introduced | 2014 |
| Researchers | A common hybrid design with no single founding paper; its components: convolution (LeCun et al. 1998) and GRU (Cho et al. 2014) |
| Original paper | Learning phrase representations using RNN encoder–decoder for statistical machine translation DOI arXiv |
| Data type | Sequences and time series |
| Domains | Time series forecasting, Audio and speech, Medical image analysis |
| Computational cost | Convolution linear in L + a GRU on L/8. |
| Data size | Small to medium. |
Problem it solves
Long signals (ECG, audio, sensors) contain important local shapes and a temporal rhythm; an RNN applied directly to thousands of samples is slow and forgets, and a CNN alone has a limited receptive field.
Background and evolution
What came before: Convolutional Neural Network, Gated recurrent unit
Core idea
Convolution with pooling turns 400 samples into 50 feature vectors, each describing a short segment, so the GRU works on a sequence eight times shorter and richer in meaning.
Interactive diagram
A simplified illustrative diagram with the dimensions of a small example, not a drawing matching every detail of the original implementation. Click any layer to see its role, dimensions and mathematical operation.
Select a layer in the diagram.
Components
| Component | Role |
|---|---|
| Conv1d + Pool | Local patterns and length reduction |
| BatchNorm | |
| (Bi)GRU | Temporal modelling |
| Head |
Equations
1. GRU (PyTorch convention)
| $z_t$ | Update gate |
| $r_t$ | Reset gate |
2. Sequence length after pooling
| $p_k$ | Size of each pooling |
How it works, step by step
- Apply 1D convolutions with pooling.
- Swap the axes to (time, channels).
- Pass through a BiGRU.
- Merge the two final states and classify.
Strengths and limitations
Strengths
- Lighter than a CNN-LSTM.
- Efficient on long signals after the reduction.
- Easy to train.
Limitations
- The choice of reduction sets the temporal resolution.
- Sequential in the recurrent part.
When to use it, and when not?
Good for
- Biosignals, audio and sensors.
Not suited for
- Very short sequences with no local structure.
Applications
| Domain | Use |
|---|---|
| Medical image analysis | ECG and EEG classification. |
| Audio and speech | Audio event detection. |
| Time series forecasting | Activity recognition from sensors. |
PyTorch code
A simplified teaching code sample that runs on small synthetic data, covering: imports, model definition, data, forward pass, loss, optimizer, training loop, evaluation and inference. It is not the paper's full implementation.
The signal is entirely synthetic, for teaching; it is not real ECG data and is not suitable for clinical conclusions.
# CNN-GRU: telling a regular rhythm from an irregular one in an ECG-like signal
# A simplified educational example in PyTorch (not the paper's full original implementation): small synthetic data,
# runs on the CPU in seconds. The numbers it prints belong to this example only.
# 1) Import the libraries
import torch
import torch.nn as nn
import torch.nn.functional as F
torch.manual_seed(0)
# 2) Define the model
class CNNGRU(nn.Module):
"""1D convolutions detect local patterns (the shape of the pulse) and reduce the length, then a GRU models the rhythm over time."""
def __init__(self, h=32, n_classes=2):
super().__init__()
self.conv = nn.Sequential(nn.Conv1d(1, 16, 7, padding=3), nn.BatchNorm1d(16), nn.ReLU(), nn.MaxPool1d(4),
nn.Conv1d(16, 32, 5, padding=2), nn.BatchNorm1d(32), nn.ReLU(), nn.MaxPool1d(2)) # 400 → 50
self.gru = nn.GRU(32, h, batch_first=True, bidirectional=True)
self.out = nn.Linear(2 * h, n_classes)
def forward(self, x): # B×L
f = self.conv(x[:, None]).transpose(1, 2) # B×50×32
_, h = self.gru(f)
return self.out(torch.cat([h[0], h[1]], 1))
# 3) Prepare the data
# An ECG-like signal (length 400): regular beats (0) or beats with irregular intervals (1), with noise and baseline drift.
# Note: synthetic data for teaching only, not real ECG signals.
def beat(L=400, pos=0):
t = torch.arange(L).float()
return torch.exp(-((t - pos) / 3) ** 2) * 1.0 - 0.25 * torch.exp(-((t - pos - 8) / 6) ** 2)
def make(n, L=400):
X = torch.zeros(n, L); y = torch.randint(0, 2, (n,))
for i in range(n):
rr = 50 + torch.randint(-5, 6, (1,)).item()
p = torch.randint(0, 30, (1,)).item()
while p < L - 10:
X[i] += beat(L, p)
p += rr if y[i] == 0 else int(rr * (0.55 + 0.9 * torch.rand(1).item()))
X[i] += 0.08 * torch.randn(L) + 0.2 * torch.sin(torch.arange(L) / 60 + torch.rand(1) * 6)
return X, y
X_train, y_train = make(800)
X_test, y_test = make(200)
model = CNNGRU()
# 4) Loss function and optimizer
loss_fn = nn.CrossEntropyLoss()
opt = torch.optim.Adam(model.parameters(), lr=3e-3)
# 5) Training loop with the forward pass
EPOCHS, BATCH = 12, 32
for epoch in range(EPOCHS):
model.train()
perm = torch.randperm(len(X_train))
total = 0.0
for i in range(0, len(X_train), BATCH):
idx = perm[i:i + BATCH]
xb, yb = X_train[idx], y_train[idx]
out = model(xb) # Forward Pass
loss = loss_fn(out, yb)
opt.zero_grad()
loss.backward() # Backpropagation
opt.step()
total += loss.item() * len(idx)
if epoch % max(1, EPOCHS // 4) == 0 or epoch == EPOCHS - 1:
print(f"epoch {epoch:3d} loss={total / len(X_train):.4f}")
# 6) Evaluation
model.eval()
with torch.no_grad():
xb = X_test
out = model(xb)
acc = (out.argmax(dim=-1) == y_test).float().mean().item()
print(f"test accuracy = {acc:.3f}")
# 7) Inference on a new sample
with torch.no_grad():
xb = X_test[:1]
out = model(xb)
print("predicted class:", out.argmax(dim=-1).item())
This code was run on 2026-11-20 in 10.0 seconds (PyTorch (CPU), Python 3 – local build machine).
Sample output:
test accuracy = 1.000
Trainable parameters in the example: model = 15,618.
These results are for small synthetic data and few epochs, and only prove that the code runs; do not compare them with published results.
Try it and go deeper
- Neural Architecture Designer
- Time Series Lab Another section of the site
Related architectures
Gated recurrent unit
Gated Recurrent Unit (GRU)
A simplification of LSTM with only two gates (update and reset) and no separate cell state, with comparable performance on many tasks and fewer parameters.
Hybrid CNN–LSTM Networks
CNN-LSTM
A CNN extracts a representation of each spatial element (a video frame, a signal window, an image), then an LSTM models the sequence of these representations over time;…
Temporal convolutional networks
Temporal Convolutional Network (TCN)
A causal 1D convolutional network (it cannot see the future) with dilated convolutions and residual connections, which processes sequences in parallel and covers a long…
Convolutional Neural Network
Convolutional Neural Network (CNN)
A network that uses small convolution filters with shared weights sliding over the image, so it captures local patterns with far fewer parameters than an MLP.
No results match the filters.
Research ideas
- A lightweight CNN-GRU for detecting atrial fibrillation on smartwatches, with an evaluation of energy consumption.
Starting suggestions, not confirmed research gaps; review the recent literature before adopting them.
Test your understanding
What is the benefit of convolution and pooling before the GRU?
Why? The GRU works on a shorter, richer sequence.
How many sets of gate weights does a GRU have compared with an LSTM?
Why? Update, reset and candidate.
A signal of length 1000 after pooling by 5 then 2: what is its length?
Why? $1000/10$.
References
Cho, K., van Merriënboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014). Learning phrase representations using RNN encoder–decoder for statistical machine translation. Proceedings of EMNLP 2014, 1724–1734. https://doi.org/10.3115/v1/D14-1179
Chung, J., Gulcehre, C., Cho, K., & Bengio, Y. (2014). Empirical evaluation of gated recurrent neural networks on sequence modeling. NIPS 2014 Workshop on Deep Learning. https://arxiv.org/abs/1412.3555
LeCun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998). Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11), 2278–2324. https://doi.org/10.1109/5.726791
