BasicTemporal and Sequential NetworksHybrid Architectures

Hybrid CNN–GRU Networks

CNN-GRU

One-dimensional convolutions capture local patterns in the signal and shorten it, then a GRU (often bidirectional) models relationships over time with fewer parameters than an LSTM.

Overview

NameHybrid CNN–GRU Networks – CNN-GRU
Year introduced2014
ResearchersA common hybrid design with no single founding paper; its components: convolution (LeCun et al. 1998) and GRU (Cho et al. 2014)
Original paperLearning phrase representations using RNN encoder–decoder for statistical machine translation DOI arXiv
Data typeSequences and time series
DomainsTime series forecasting, Audio and speech, Medical image analysis
Computational costConvolution linear in L + a GRU on L/8.
Data sizeSmall to medium.

Problem it solves

Long signals (ECG, audio, sensors) contain important local shapes and a temporal rhythm; an RNN applied directly to thousands of samples is slow and forgets, and a CNN alone has a limited receptive field.

Background and evolution

What came before: Convolutional Neural Network, Gated recurrent unit

  1. 2014

    Cho and colleagues introduced the GRU. [ref]

  2. 2014

    Chung and colleagues compared GRU and LSTM empirically and found similar performance on their tasks. [ref]

Core idea

Convolution with pooling turns 400 samples into 50 feature vectors, each describing a short segment, so the GRU works on a sequence eight times shorter and richer in meaning.

Example: Detecting cardiac arrhythmia: the convolution recognises the shape of a beat, and the GRU compares the intervals between beats.

Interactive diagram

A simplified illustrative diagram with the dimensions of a small example, not a drawing matching every detail of the original implementation. Click any layer to see its role, dimensions and mathematical operation.

Select a layer in the diagram.

Components

ComponentRole
Conv1d + PoolLocal patterns and length reduction
BatchNorm
(Bi)GRUTemporal modelling
Head

Equations

1. GRU (PyTorch convention)

$$z_t=\sigma(W_zx_t+U_zh_{t-1}),\;r_t=\sigma(W_rx_t+U_rh_{t-1}),\;n_t=\tanh(W_nx_t+r_t\odot U_nh_{t-1}),\;h_t=(1-z_t)\odot n_t+z_t\odot h_{t-1}$$

Symbols
$z_t$Update gate
$r_t$Reset gate
Dimensions
$h_t\in\mathbb{R}^{h}$
Numerical example
$z=0.9$, $n=1$, $h_{t-1}=0$ ⇒ $h_t=0.1$: the gate mostly keeps the old state.
Relation to the architecture
Three sets of weights instead of four in an LSTM. (Cho's paper uses the role of z reversed.)

2. Sequence length after pooling

$$L_{out}=\left\lfloor\frac{L}{\prod_k p_k}\right\rfloor$$

Symbols
$p_k$Size of each pooling
Dimensions
An integer
Numerical example
$L=400$ with pooling of 4 then 2: $L_{out}=50$.
Relation to the architecture
It reduces the cost of the recurrent part.

How it works, step by step

  1. Apply 1D convolutions with pooling.
  2. Swap the axes to (time, channels).
  3. Pass through a BiGRU.
  4. Merge the two final states and classify.

Strengths and limitations

Strengths

  • Lighter than a CNN-LSTM.
  • Efficient on long signals after the reduction.
  • Easy to train.

Limitations

  • The choice of reduction sets the temporal resolution.
  • Sequential in the recurrent part.

When to use it, and when not?

Good for

  • Biosignals, audio and sensors.

Not suited for

  • Very short sequences with no local structure.

Applications

DomainUse
Medical image analysisECG and EEG classification.
Audio and speechAudio event detection.
Time series forecastingActivity recognition from sensors.

PyTorch code

A simplified teaching code sample that runs on small synthetic data, covering: imports, model definition, data, forward pass, loss, optimizer, training loop, evaluation and inference. It is not the paper's full implementation.

The signal is entirely synthetic, for teaching; it is not real ECG data and is not suitable for clinical conclusions.

Python (PyTorch)
# CNN-GRU: telling a regular rhythm from an irregular one in an ECG-like signal
# A simplified educational example in PyTorch (not the paper's full original implementation): small synthetic data,
# runs on the CPU in seconds. The numbers it prints belong to this example only.
# 1) Import the libraries
import torch
import torch.nn as nn
import torch.nn.functional as F

torch.manual_seed(0)

# 2) Define the model
class CNNGRU(nn.Module):
    """1D convolutions detect local patterns (the shape of the pulse) and reduce the length, then a GRU models the rhythm over time."""
    def __init__(self, h=32, n_classes=2):
        super().__init__()
        self.conv = nn.Sequential(nn.Conv1d(1, 16, 7, padding=3), nn.BatchNorm1d(16), nn.ReLU(), nn.MaxPool1d(4),
                                  nn.Conv1d(16, 32, 5, padding=2), nn.BatchNorm1d(32), nn.ReLU(), nn.MaxPool1d(2))   # 400 → 50
        self.gru = nn.GRU(32, h, batch_first=True, bidirectional=True)
        self.out = nn.Linear(2 * h, n_classes)
    def forward(self, x):                                      # B×L
        f = self.conv(x[:, None]).transpose(1, 2)              # B×50×32
        _, h = self.gru(f)
        return self.out(torch.cat([h[0], h[1]], 1))

# 3) Prepare the data
# An ECG-like signal (length 400): regular beats (0) or beats with irregular intervals (1), with noise and baseline drift.
# Note: synthetic data for teaching only, not real ECG signals.
def beat(L=400, pos=0):
    t = torch.arange(L).float()
    return torch.exp(-((t - pos) / 3) ** 2) * 1.0 - 0.25 * torch.exp(-((t - pos - 8) / 6) ** 2)
def make(n, L=400):
    X = torch.zeros(n, L); y = torch.randint(0, 2, (n,))
    for i in range(n):
        rr = 50 + torch.randint(-5, 6, (1,)).item()
        p = torch.randint(0, 30, (1,)).item()
        while p < L - 10:
            X[i] += beat(L, p)
            p += rr if y[i] == 0 else int(rr * (0.55 + 0.9 * torch.rand(1).item()))
        X[i] += 0.08 * torch.randn(L) + 0.2 * torch.sin(torch.arange(L) / 60 + torch.rand(1) * 6)
    return X, y
X_train, y_train = make(800)
X_test, y_test = make(200)
model = CNNGRU()

# 4) Loss function and optimizer
loss_fn = nn.CrossEntropyLoss()
opt = torch.optim.Adam(model.parameters(), lr=3e-3)

# 5) Training loop with the forward pass
EPOCHS, BATCH = 12, 32
for epoch in range(EPOCHS):
    model.train()
    perm = torch.randperm(len(X_train))
    total = 0.0
    for i in range(0, len(X_train), BATCH):
        idx = perm[i:i + BATCH]
        xb, yb = X_train[idx], y_train[idx]
        out = model(xb)                       # Forward Pass
        loss = loss_fn(out, yb)
        opt.zero_grad()
        loss.backward()                    # Backpropagation
        opt.step()
        total += loss.item() * len(idx)
    if epoch % max(1, EPOCHS // 4) == 0 or epoch == EPOCHS - 1:
        print(f"epoch {epoch:3d}  loss={total / len(X_train):.4f}")

# 6) Evaluation
model.eval()
with torch.no_grad():
    xb = X_test
    out = model(xb)
    acc = (out.argmax(dim=-1) == y_test).float().mean().item()
    print(f"test accuracy = {acc:.3f}")

# 7) Inference on a new sample
with torch.no_grad():
    xb = X_test[:1]
    out = model(xb)
    print("predicted class:", out.argmax(dim=-1).item())

This code was run on 2026-11-20 in 10.0 seconds (PyTorch (CPU), Python 3 – local build machine).

Sample output:

test accuracy = 1.000

Trainable parameters in the example: model = 15,618.

These results are for small synthetic data and few epochs, and only prove that the code runs; do not compare them with published results.

Try it and go deeper

Related architectures

Basic2014

Gated recurrent unit

Gated Recurrent Unit (GRU)

A simplification of LSTM with only two gates (update and reset) and no separate cell state, with comparable performance on many tasks and fewer parameters.

Temporal and Sequential Networks

Basic2015

Hybrid CNN–LSTM Networks

CNN-LSTM

A CNN extracts a representation of each spatial element (a video frame, a signal window, an image), then an LSTM models the sequence of these representations over time;…

Temporal and Sequential NetworksHybrid Architectures

Basic2018

Temporal convolutional networks

Temporal Convolutional Network (TCN)

A causal 1D convolutional network (it cannot see the future) with dilated convolutions and residual connections, which processes sequences in parallel and covers a long…

Temporal and Sequential Networks

Beginner1998

Convolutional Neural Network

Convolutional Neural Network (CNN)

A network that uses small convolution filters with shared weights sliding over the image, so it captures local patterns with far fewer parameters than an MLP.

Foundations

Research ideas

  • A lightweight CNN-GRU for detecting atrial fibrillation on smartwatches, with an evaluation of energy consumption.

Starting suggestions, not confirmed research gaps; review the recent literature before adopting them.

Test your understanding

  1. Concept

    What is the benefit of convolution and pooling before the GRU?

  2. Comparison

    How many sets of gate weights does a GRU have compared with an LSTM?

  3. Concept

    A signal of length 1000 after pooling by 5 then 2: what is its length?

References

  1. Cho, K., van Merriënboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014). Learning phrase representations using RNN encoder–decoder for statistical machine translation. Proceedings of EMNLP 2014, 1724–1734. https://doi.org/10.3115/v1/D14-1179

    DOIarXivAutomatically verified via Crossref – 2026-11-20

  2. Chung, J., Gulcehre, C., Cho, K., & Bengio, Y. (2014). Empirical evaluation of gated recurrent neural networks on sequence modeling. NIPS 2014 Workshop on Deep Learning. https://arxiv.org/abs/1412.3555

    arXivAutomatically verified via OpenAlex/arXiv – 2026-11-20

  3. LeCun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998). Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11), 2278–2324. https://doi.org/10.1109/5.726791

    DOIAutomatically verified via Crossref – 2026-11-20