Bagging trains many models on bootstrap samples (drawn with replacement) and averages them, which reduces variance. Random forests are bagged decision trees.
Implement with NumPy:
bootstrap_indices(n, n_models, rng): an (n_models, n) integer array, each row drawn with rng.integers(0, n, n)fit_stump(X, y): a depth-1 regression tree. Try every feature and every threshold halfway between consecutive sorted unique values; pick the split minimising total squared error (ties: lowest feature, then lowest threshold). Return (feature, threshold, left_mean, right_mean); if no split is possible, return (0, inf, mean, mean).predict_stump(stump, X): left if X[:, f] <= tbagged_predict(X_train, y_train, X_test, n_models, rng): fit one stump per bootstrap sample and average their predictionsoob_score(X, y, n_models, rng): the out-of-bag MSE. Each training point is predicted only by the models whose bootstrap sample excluded it; points never left out are skipped.