Split Conformal Prediction

medium
Unit 16 · Semi Supervised0/3 to milestone

Conformal prediction wraps any model with intervals that have a guaranteed coverage of at least 1 − α, assuming only that the data points are exchangeable.

Using a held-out calibration set:

  • conformal_quantile(scores, alpha): with n calibration scores, return the k-th smallest where k=⌈(n+1)(1−α)⌉k = \lceil (n + 1)(1 - \alpha) \rceil, or inf if k > n
  • regression_intervals(cal_pred, cal_y, test_pred, alpha): scores are |y - pred|. Return (lower, upper) arrays: test_pred ∓ q.
  • classification_sets(cal_probs, cal_y, test_probs, alpha): scores are 1 - p(true class). Each test set is the sorted list of classes c with 1 - p_c <= q.
  • coverage(sets_or_intervals, y): the fraction of test points whose true value is covered. It accepts either a (lower, upper) pair of arrays or a list of sets.
#uncertainty#conformal#calibration2 visible + 1 hidden tests
solution.py · Python 3.13
Loading editor…
Run your code to see results. ⌘↵ to run · ⌘⇧↵ to submit
Split Conformal Prediction · TechGradify