Adagrad

easy
Unit 7 · NLP0/3 to milestone

Adagrad gives each parameter its own learning rate, shrinking it for parameters that have seen large gradients:

G←G+g2,θ←θ−ηG+ϵ gG \leftarrow G + g^2,\qquad \theta \leftarrow \theta - \frac{\eta}{\sqrt{G} + \epsilon}\,g

Implement adagrad_step(theta, g, G, lr=0.1, eps=1e-10) returning (theta_new, G_new) without modifying the inputs.

#optimizers2 visible + 1 hidden tests
solution.py · Python 3.13
Loading editor…
Run your code to see results. ⌘↵ to run · ⌘⇧↵ to submit
Adagrad · TechGradify