Spiking Neural Networks (SNN)
Networks whose neurons communicate with discrete binary spikes in time, like the brain. The membrane potential accumulates and fires a spike when it exceeds the threshold, and is then reset.
Overview
| Name | Spiking Neural Networks |
|---|---|
| Year introduced | 1997 |
| Researchers | Wolfgang Maass (third-generation framing); training with surrogate gradients: Neftci, Mostafa, Zenke (2019) |
| Original paper | Networks of spiking neurons: The third generation of neural network models DOI |
| Data type | Sequences and time series, Temporal events and spikes |
| Domains | Computer vision, Robotics, Audio and speech |
| Computational cost | On a GPU: the cost of T time steps; on neuromorphic hardware: event-driven computation. |
| Data size | Medium; often used with event data. |
Problem it solves
Classic networks consume a lot of energy because they compute continuous values for every unit at every step. Neuromorphic computing aims at event-driven computation that consumes less.
Background and evolution
What came before: Multilayer Perceptron, Hodgkin–Huxley, Integrate-and-Fire
Core idea
In the LIF model the potential leaks towards the resting value and rises with the input current. When it reaches the threshold $V_{th}$ a spike $S=1$ is fired and the potential is reset. The spike function (a Heaviside step) is not differentiable, so its gradient is replaced in backpropagation by a smooth function (a surrogate).
Interactive diagram
A simplified illustrative diagram with the dimensions of a small example, not a drawing matching every detail of the original implementation. Click any layer to see its role, dimensions and mathematical operation.
Select a layer in the diagram.
Components
| Component | Role |
|---|---|
| Input encoding | Rate (Poisson) or temporal encoding, or event data |
| LIF neurons | A membrane potential that leaks and accumulates |
| Threshold & reset | Firing a spike and resetting |
| Synaptic weights | Weights between layers |
| Surrogate gradient | Approximating the spike derivative |
| Readout | Spike counts or the output layer potential |
Equations
1. Continuous LIF equation
| $\tau_m$ | Membrane time constant |
| $V_{rest}$ | Resting potential |
| $R$ | Resistance |
| $I(t)$ | Input current |
2. The discrete form with firing
| $\beta$ | Leak factor |
| $\Theta$ | Step function |
| $S_t$ | The spike is 0 or 1 |
3. Surrogate gradient
| $k$ | Steepness of the curve |
How it works, step by step
- The input is encoded as spike trains over T steps.
- At each step the current is computed and the potential updated.
- Spikes are fired and the potential reset.
- The output spikes are counted.
- The gradient through time is computed with the smooth surrogate.
Strengths and limitations
Strengths
- Potential energy efficiency on neuromorphic hardware.
- Well suited to event data (event cameras).
- Natural temporal modelling.
Limitations
- Training is harder and slower on ordinary GPUs.
- Accuracy is often lower than equivalent classic networks.
- Energy gains are realised on specialised hardware.
When to use it, and when not?
Good for
- Low-power embedded systems.
- Event sensor data.
Not suited for
- When top accuracy on a conventional GPU is the priority.
Applications
| Domain | Use |
|---|---|
| Robotics | Low-power control in robotics. |
| Audio and speech | Keyword spotting. |
| Computer vision | Processing event-camera data. |
PyTorch code
A simplified teaching code sample that runs on small synthetic data, covering: imports, model definition, data, forward pass, loss, optimizer, training loop, evaluation and inference. It is not the paper's full implementation.
Libraries such as snnTorch or SpikingJelly can be used for larger models; here is a hand-written implementation to illustrate the equations.
# Spiking Neural Network with LIF neurons and a surrogate gradient
# A simplified educational example in PyTorch (not the paper's full original implementation): small synthetic data,
# runs on the CPU in seconds. The numbers it prints belong to this example only.
# 1) Import the libraries
import torch
import torch.nn as nn
import torch.nn.functional as F
torch.manual_seed(0)
# 2) Define the model
class SpikeFn(torch.autograd.Function):
@staticmethod
def forward(ctx, v):
ctx.save_for_backward(v)
return (v > 0).float() # Θ(V - V_th) after the shift
@staticmethod
def backward(ctx, g):
v, = ctx.saved_tensors
return g / (1 + 10 * v.abs()) ** 2 # surrogate gradient
spike = SpikeFn.apply
class SNN(nn.Module):
def __init__(self, d_in=20, h=64, n_classes=2, beta=0.9, T=25):
super().__init__()
self.fc1, self.fc2, self.beta, self.T = nn.Linear(d_in, h), nn.Linear(h, n_classes), beta, T
def forward(self, x): # x: B×d in [0,1] ⇒ rate encoding
B = x.shape[0]
v1, v2 = torch.zeros(B, self.fc1.out_features), torch.zeros(B, self.fc2.out_features)
count = torch.zeros(B, self.fc2.out_features)
for t in range(self.T):
s_in = (torch.rand_like(x) < x).float() # Poisson spikes
v1 = self.beta * v1 + self.fc1(s_in); s1 = spike(v1 - 1.0); v1 = v1 - s1 # reset by subtraction
v2 = self.beta * v2 + self.fc2(s1); s2 = spike(v2 - 1.0); v2 = v2 - s2
count = count + s2
return count # spike count of each class
# 3) Prepare the data
# Two classes: a high-intensity pattern in the first half of the inputs or in the second half
y = torch.randint(0, 2, (600,))
X = torch.rand(600, 20) * 0.3
X[y == 0, :10] += 0.6; X[y == 1, 10:] += 0.6
X = X.clamp(0, 1)
X_train, y_train, X_test, y_test = X[:480], y[:480], X[480:], y[480:]
model = SNN()
# 4) Loss function and optimizer
loss_fn = nn.CrossEntropyLoss()
opt = torch.optim.Adam(model.parameters(), lr=5e-3)
# 5) Training loop with the forward pass
EPOCHS, BATCH = 8, 32
for epoch in range(EPOCHS):
model.train()
perm = torch.randperm(len(X_train))
total = 0.0
for i in range(0, len(X_train), BATCH):
idx = perm[i:i + BATCH]
xb, yb = X_train[idx], y_train[idx]
out = model(xb) # Forward Pass
loss = loss_fn(out, yb)
opt.zero_grad()
loss.backward() # Backpropagation
opt.step()
total += loss.item() * len(idx)
if epoch % max(1, EPOCHS // 4) == 0 or epoch == EPOCHS - 1:
print(f"epoch {epoch:3d} loss={total / len(X_train):.4f}")
# 6) Evaluation
model.eval()
with torch.no_grad():
xb = X_test
out = model(xb)
acc = (out.argmax(dim=-1) == y_test).float().mean().item()
print(f"test accuracy = {acc:.3f}")
# 7) Inference on a new sample
with torch.no_grad():
xb = X_test[:1]
out = model(xb)
print("predicted class:", out.argmax(dim=-1).item())
This code was run on 2026-11-20 in 3.5 seconds (PyTorch (CPU), Python 3 – local build machine).
Sample output:
test accuracy = 1.000
Trainable parameters in the example: model = 1,474.
These results are for small synthetic data and few epochs, and only prove that the code runs; do not compare them with published results.
Try it and go deeper
Related architectures
Liquid Neural Networks
Liquid Neural Networks (LNN)
Continuous-time recurrent networks in which each neuron's time constant changes with the input (Liquid Time-Constant), so their dynamics adapt to the data with a small n…
Reservoir Computing
Reservoir Computing (ESN / LSM)
A large recurrent network with fixed random weights (the reservoir) turns the input into rich states, and only the linear readout layer is trained, usually with closed-f…
Recurrent neural network
Recurrent Neural Network (RNN)
A network that processes a sequence element by element and carries a hidden state that passes information from previous steps, using the same weights at every step.
No results match the filters.
Research ideas
- Estimate the actual energy consumption of an SNN on neuromorphic hardware versus a quantised CNN for an Arabic keyword-spotting task.
Starting suggestions, not confirmed research gaps; review the recent literature before adopting them.
Test your understanding
When does an LIF neuron fire a spike?
Why? Firing requires $V_t\ge V_{th}$.
Why do we need a surrogate gradient?
Why? We replace the derivative with a smooth function during backpropagation only.
With $\beta=0.5$, $V_{t-1}=1.2$, an input of $0.5$ and a threshold of 1, does the neuron fire?
Why? $0.5\times1.2+0.5=1.1>1$.
References
Maass, W. (1997). Networks of spiking neurons: The third generation of neural network models. Neural Networks, 10(9), 1659–1671. https://doi.org/10.1016/S0893-6080(97)00011-7
Neftci, E. O., Mostafa, H., & Zenke, F. (2019). Surrogate gradient learning in spiking neural networks: Bringing the power of gradient-based optimization to spiking neural networks. IEEE Signal Processing Magazine, 36(6), 51–63. https://doi.org/10.1109/MSP.2019.2931595
Gerstner, W., Kistler, W. M., Naud, R., & Paninski, L. (2014). Neuronal dynamics: From single neurons to networks and models of cognition. Cambridge University Press. https://doi.org/10.1017/CBO9781107447615
