Plain gradient descent zig-zags across narrow valleys. Momentum keeps a running velocity so consistent directions speed up and oscillations cancel out.
With learning rate and momentum (PyTorch's formulation):
Implement sgd_step(w, g, v, lr, momentum=0.0, nesterov=False) returning (new_w, new_v) without modifying the inputs. Then implement minimize(grad_fn, w0, lr, momentum, nesterov, steps), which runs the optimiser from v = 0 and returns the list of positions, including w0.