The expected test error of a model splits into bias² (how far the average prediction is from the truth), variance (how much predictions move between training sets) and irreducible noise. Measure it by simulating many training sets.
Implement bias_variance(f, noise_sd, degree, n_train, n_sims, x_test, rng):
x = rng.uniform(0, 1, n_train), then y = f(x) + rng.normal(0, noise_sd, n_train) (in that order), fit np.polyfit(x, y, degree) and predict at x_test."bias2" (the mean over test points of (mean_prediction - f(x_test))²), "variance" (the mean over test points of the variance of predictions across simulations) and "noise" (noise_sd²).Also implement expected_mse(result): bias2 + variance + noise.