Gradient Checking

Before trusting a hand-written backward pass, compare it with a numerical gradient from central differences: $\frac{f(x + h) - f(x - h)}{2h}$ for each element.

Implement:

• numerical_grad(f, x, h=1e-5): f maps an array to a scalar. Return an array shaped like x. Perturb one element at a time and restore it afterwards, so x is unchanged when you return.
• relative_error(a, b): the maximum over elements of |a - b| / max(|a| + |b|, 1e-12)
• check_gradient(f, grad_f, x, tol=1e-6): return (ok, error), comparing grad_f(x) with the numerical gradient