Tutorials and simulations
Play with the ideas before you code them. Drag the sliders, step through the algorithm, and then solve the linked problems.
Gradient descent
How models learn: walk downhill on the loss, one step at a time.
Step 0 · x = -2.400 · loss = 10.878 · gradient = -12.424
Each step moves against the slope: x ← x − lr · ∇loss. Try a tiny learning rate (slow), a big one (it overshoots or diverges), and starting on the left, where plain descent gets stuck in the local minimum. Momentum can carry it over the bump.