ExperimentAdvancedQuantum machine learning

Comparing Optimizers: GD, Adam and SPSA

The same classifier from the same start is trained with three optimizers, comparing loss and number of function evaluations; SPSA needs only two evaluations per step regardless of parameter count.

Goal

Compare the behaviour of the optimisers and their cost in number of evaluations.

Approximate duration: 8 minutes

Steps

  1. Choose blobs, two layers and 40 steps.
  2. Run the comparison.
  3. Compare the loss curves.
  4. Compare the number of evaluations for each optimiser.
Loading the simulator…
What theory predicts

Theory predicts that the GD and Adam gradient needs $2p=16$ evaluations per step ($p=8$ parameters), and SPSA only two; with one extra evaluation per step to record the loss, the counter after 40 steps is 681 evaluations each for GD and Adam against 121 for SPSA. The SPSA curve is expected to oscillate more; the final ordering of the losses depends on the seed and the learning rate.

Gradient descent (GD) and Adam use the full gradient; with the parameter-shift rule each gradient costs $2p$ circuit evaluations for $p$ parameters. SPSA (Spall 1992) estimates the gradient from only two evaluations in a random direction; it suits noisy hardware, but its path oscillates more.

On hardware each evaluation is measured with a finite number of shots, so the cost of evaluations is the real resource. In deep random circuits gradients may vanish exponentially (Barren Plateaus), and changing the optimiser does not by itself solve this problem.

Test your understanding

The quiz needs JavaScript.

Related topics

References

  1. J. C. Spall (1992). Multivariate stochastic approximation using a simultaneous perturbation gradient approximation. IEEE Trans. Automatic Control 37(3):332–341 doi:10.1109/9.119632
  2. Diederik P. Kingma; Jimmy Ba (2014). Adam: A Method for Stochastic Optimization. arXiv:1412.6980 doi:10.48550/arXiv.1412.6980
  3. Jarrod R. McClean; Sergio Boixo; Vadim N. Smelyanskiy; Ryan Babbush; Hartmut Neven (2018). Barren plateaus in quantum neural network training landscapes. Nature Communications 9:4812 doi:10.1038/s41467-018-07090-4
  4. M. Cerezo et al. (2021). Variational quantum algorithms. Nature Reviews Physics 3:625–644 doi:10.1038/s42254-021-00348-9