Adagrad gives each parameter its own learning rate, shrinking it for parameters that have seen large gradients:
Implement adagrad_step(theta, g, G, lr=0.1, eps=1e-10) returning (theta_new, G_new) without modifying the inputs.
adagrad_step(theta, g, G, lr=0.1, eps=1e-10)
(theta_new, G_new)