RMSprop Optimiser

easy
Unit 3 · Optimization0/3 to milestone

RMSprop adapts the step size per parameter: it divides each gradient by a running root-mean-square of its recent magnitudes, so parameters with large, noisy gradients take smaller steps.

s←ρs+(1−ρ)g2,w←w−η gs+ϵs \leftarrow \rho s + (1 - \rho) g^2, \qquad w \leftarrow w - \frac{\eta\, g}{\sqrt{s} + \epsilon}

Implement rmsprop_step(w, g, s, lr=0.01, rho=0.9, eps=1e-8) returning (new_w, new_s) without modifying the inputs, and minimize_rmsprop(grad_fn, w0, steps, lr=0.01), which starts from s = 0 and returns the final w.

#optimization#training2 visible + 2 hidden tests
solution.py · Python 3.13
Loading editor…
Run your code to see results. ⌘↵ to run · ⌘⇧↵ to submit
RMSprop Optimiser · TechGradify