Greedy speculative decoding only works for argmax decoding. Speculative sampling keeps the exact output distribution of the big model when sampling. For each drafted token , with draft probability and target probability :
Unlock this and every premium problem across all tracks — plus hidden-test grading, advanced AI engineering modules and new content every week.
Start 7-day free trial