Bidirectional LSTM (BiLSTM)
Two recurrent layers, one reading the sequence from the start and the other from the end; their states are concatenated so that every position sees both its preceding and following context.
Overview
| Name | Bidirectional LSTM |
|---|---|
| Year introduced | 1997 |
| Researchers | Mike Schuster, Kuldip K. Paliwal (bidirectional RNN); Alex Graves, Jürgen Schmidhuber (BiLSTM, 2005) |
| Original paper | Bidirectional recurrent neural networks DOI |
| Data type | Sequences and time series, Text |
| Domains | Audio and speech, Medical image analysis, Natural language processing |
Problem it solves
In tasks such as word tagging or phoneme classification, the meaning of an element also depends on what follows it, and a unidirectional network cannot see the future.
Background and evolution
What came before: Recurrent neural network, Long short-term memory
Core idea
A forward state $\overrightarrow{h}_t$ is computed from the start up to t, and a backward state $\overleftarrow{h}_t$ from the end down to t, and the two are concatenated. It is unsuitable for live forecasting because it needs the whole sequence.
Interactive diagram
A simplified illustrative diagram with the dimensions of a small example, not a drawing matching every detail of the original implementation. Click any layer to see its role, dimensions and mathematical operation.
Select a layer in the diagram.
Components
| Component | Role |
|---|---|
| Forward LSTM | Left to right |
| Backward LSTM | Right to left |
| Concatenation | Concatenating the two states |
| Output layer | Per position or per sequence |
Equations
1. Concatenating both directions
| $\overrightarrow{h}_t$ | Forward state |
| $\overleftarrow{h}_t$ | Backward state |
How it works, step by step
- The forward network passes over the sequence.
- The backward network passes in reverse.
- The two states are concatenated for every position.
- The classifier is applied to every position or to a pooled summary.
Strengths and limitations
Strengths
- Full context for every position.
- Clear gains in tagging and speech recognition tasks.
Limitations
- Unsuitable for real-time prediction or step-by-step generation.
- Twice the compute of a single LSTM.
When to use it, and when not?
Good for
- Word tagging and entity recognition.
- Classifying complete recorded signals.
Not suited for
- Real-time forecasting of the future.
Applications
| Domain | Use |
|---|---|
| Natural language processing | Named-entity recognition and part-of-speech tagging. |
| Audio and speech | Frame-level phoneme classification. |
| Medical image analysis | Classifying recorded ECG beats. |
PyTorch code
A simplified teaching code sample that runs on small synthetic data, covering: imports, model definition, data, forward pass, loss, optimizer, training loop, evaluation and inference. It is not the paper's full implementation.
# Bidirectional LSTM — per-position tagging
# A simplified educational example in PyTorch (not the paper's full original implementation): small synthetic data,
# runs on the CPU in seconds. The numbers it prints belong to this example only.
# 1) Import the libraries
import torch
import torch.nn as nn
import torch.nn.functional as F
torch.manual_seed(0)
# 2) Define the model
class BiLSTMTagger(nn.Module):
def __init__(self, d_in=1, h=32, n_tags=2):
super().__init__()
self.lstm = nn.LSTM(d_in, h, batch_first=True, bidirectional=True)
self.out = nn.Linear(2 * h, n_tags)
def forward(self, x):
H, _ = self.lstm(x) # B×T×2h
return self.out(H).transpose(1, 2) # B×tags×T (CrossEntropy shape for sequences)
# 3) Prepare the data
# Correct tag for each position: is the next value larger than the current one? (requires seeing the future)
X = torch.randn(800, 20, 1)
nxt = torch.cat([X[:, 1:, 0], X[:, -1:, 0]], 1)
Y = (nxt > X[:, :, 0]).long()
X_train, y_train, X_test, y_test = X[:640], Y[:640], X[640:], Y[640:]
model = BiLSTMTagger()
# 4) The loss and the optimizer
loss_fn = nn.CrossEntropyLoss()
opt = torch.optim.Adam(model.parameters(), lr=1e-2)
# 5) Training loop
for epoch in range(15):
model.train()
for i in range(0, len(X_train), 64):
loss = loss_fn(model(X_train[i:i + 64]), y_train[i:i + 64])
opt.zero_grad(); loss.backward(); opt.step()
if epoch % 5 == 0: print(f"epoch {epoch} loss={loss.item():.4f}")
# 6) Evaluation (position-level accuracy)
model.eval()
with torch.no_grad():
acc = (model(X_test).argmax(1) == y_test).float().mean().item()
print(f"test accuracy = {acc:.3f}")
# 7) Inference: tags of the first 10 positions
with torch.no_grad():
print("tags:", model(X_test[:1]).argmax(1)[0, :10].tolist())
This code was run on 2026-11-20 in 3.5 seconds (PyTorch (CPU), Python 3 – local build machine).
Sample output:
test accuracy = 0.989
Trainable parameters in the example: model = 9,090.
These results are for small synthetic data and few epochs, and only prove that the code runs; do not compare them with published results.
Try it and go deeper
- AI Model Atlas Another section of the site
- Natural Language Processing Lab Another section of the site
Related architectures
Long short-term memory
Long Short-Term Memory (LSTM)
A recurrent network with a memory cell and gates (forget, input, output) that control what is kept and what is erased, so it learns long-term dependencies far better tha…
Gated recurrent unit
Gated Recurrent Unit (GRU)
A simplification of LSTM with only two gates (update and reset) and no separate cell state, with comparable performance on many tasks and fewer parameters.
Transformer
Transformer
An architecture based entirely on multi-head self-attention instead of recurrence or convolution, so it processes all sequence positions in parallel and links any two po…
No results match the filters.
Research ideas
- BiLSTM-CRF vs small Transformer models for entity recognition in Arabic medical texts.
Starting suggestions, not confirmed research gaps; review the recent literature before adopting them.
Test your understanding
Why is BiLSTM unsuitable for real-time forecasting of the future?
Why? Reading from the end requires the end to exist.
What is the per-position output size of a BiLSTM with hidden state h=128 per direction?
Why? Concatenating two states of 128 each.
In which task does BiLSTM usually beat a unidirectional LSTM?
Why? A word's meaning depends on what comes before and after it.
References
Schuster, M., & Paliwal, K. K. (1997). Bidirectional recurrent neural networks. IEEE Transactions on Signal Processing, 45(11), 2673–2681. https://doi.org/10.1109/78.650093
Graves, A., & Schmidhuber, J. (2005). Framewise phoneme classification with bidirectional LSTM and other neural network architectures. Neural Networks, 18(5–6), 602–610. https://doi.org/10.1016/j.neunet.2005.06.042
Hochreiter, S., & Schmidhuber, J. (1997). Long short-term memory. Neural Computation, 9(8), 1735–1780. https://doi.org/10.1162/neco.1997.9.8.1735
