ConceptIntermediatequantum machine learningLesson 7

Hybrid Quantum–Classical Learning

Hybrid learning is a loop in which a quantum processor evaluates the model and its gradients from measurements while a classical computer updates the parameters with an optimizer such as GD, Adam or SPSA.

All variational models in quantum machine learning are hybrid quantum–classical: the quantum computer prepares and measures states, and the classical computer computes the loss and updates the parameters. Each training step consists of: (1) for each training sample, running the circuit for a number of shots and estimating $\langle O\rangle$; (2) converting the value into a prediction, such as $p=(1-\langle Z_0\rangle)/2$; (3) computing the loss, such as the binary cross-entropy $$\mathcal{L}=-\frac{1}{m}\sum_{i=1}^m\big[y_i\ln p_i+(1-y_i)\ln(1-p_i)\big];$$ (4) estimating the gradient and updating $\theta$.

Gradient estimation on quantum hardware is not done with reverse-mode automatic differentiation as in classical networks, because the intermediate states are not accessible. The standard method is the parameter-shift rule, which gives the exact derivative for each parameter from two evaluations of the same circuit with shifts of $\pm\pi/2$; a full gradient with $P$ parameters needs $2P$ evaluations. PennyLane provides an integration that makes circuits differentiable within machine learning frameworks and chooses the differentiation method suited to the device.

Common optimisers: plain gradient descent; Adam, which adapts the step size for each parameter using moving estimates of the first and second moments; and SPSA (simultaneous perturbation stochastic approximation), which estimates the gradient from only two evaluations whatever the number of parameters, by shifting all parameters together in a random direction; it suits noisy measurements but gives a random estimate of the gradient. The lab lets you compare the three.

The most important practical consideration is the shot budget: each evaluation needs $S$ shots, and the error falls as $1/\sqrt{S}$, so gradient accuracy and cost pull against each other. On devices there is also the latency of moving between the two computers. That is why mini-batches, decaying learning rates and careful initialisation are used. The lab's simulation computes the expectation values exactly without shot noise, so the training curves look smoother than they would on a real device.

A common mistake when designing hybrid experiments is to compare a carefully trained quantum model with an untuned classical one; a fair comparison needs a similar number of parameters and an equivalent training budget.

Mathematical formula

$$\theta_{t+1}=\theta_t-\eta\,\nabla_\theta\mathcal{L}(\theta_t),\qquad \mathcal{L}=-\frac{1}{m}\sum_i\big[y_i\ln p_i+(1-y_i)\ln(1-p_i)\big]$$

Worked example

  1. The loss: one sample with $y=1$, and the model gave $\langle Z_0\rangle=-0.6$, so $p=(1+0.6)/2=0.8$ and $\mathcal{L}=-\ln0.8\approx0.223$. Had it given $p=0.2$, the loss would be $-\ln0.2\approx1.609$.
  1. Gradient cost: a model with $P=8$ parameters, $S=1000$ shots per evaluation and one sample. Parameter shift: $2\times8=16$ evaluations, i.e. $16\,000$ shots per step. SPSA: only two evaluations, i.e. $2\,000$ shots, but with a noisier gradient estimate.

Test your understanding

The quiz needs JavaScript.

Related topics

References

  1. Ville Bergholm et al. (2018). PennyLane: Automatic differentiation of hybrid quantum-classical computations. arXiv:1811.04968 doi:10.48550/arXiv.1811.04968
  2. J. C. Spall (1992). Multivariate stochastic approximation using a simultaneous perturbation gradient approximation. IEEE Trans. Automatic Control 37(3):332–341 doi:10.1109/9.119632
  3. Diederik P. Kingma; Jimmy Ba (2014). Adam: A Method for Stochastic Optimization. arXiv:1412.6980 doi:10.48550/arXiv.1412.6980