Adagrad is a deep learning optimization algorithm that adapts the learning rate for each weight in the model. The idea is that weights that are updated frequently get a progressively smaller learning rate, while weights updated less often keep a larger one, which helps with data of varied distribution and rare features. Despite this automatic adaptation, one of its drawbacks is that the learning rate can shrink too much over time, which slows or stops learning.
