A cascade sends every query to a cheap model first and escalates to an expensive one only when the cheap model isn't confident. A good threshold keeps most of the big model's accuracy at a fraction of the cost (FrugalGPT).
Each query has (conf, small_ok, large_ok): the small model's confidence and whether each model answered correctly. With a threshold τ, keep the small answer if conf >= τ; otherwise you pay for both models and use the large one's answer.
Implement:
evaluate(queries, tau, c_small, c_large): return (accuracy, mean_cost)frontier(queries, c_small, c_large): evaluate every threshold in sorted(set(confs)) + [inf], and return the Pareto-optimal (tau, accuracy, cost) points (no other point is at least as accurate and at least as cheap with one of them strictly better), sorted by costcheapest_for(queries, c_small, c_large, target_acc): the lowest-cost (tau, accuracy, cost) whose accuracy reaches target_acc, or None