During generation, a Transformer caches the keys and values for every previous token. Serving engineers must budget GPU memory for it.
Unlock this and every premium problem across all tracks — plus hidden-test grading, advanced AI engineering modules and new content every week.
Start 7-day free trial