In grouped-query attention, many query heads share fewer key/value heads. Llama 3 70B uses 64 query heads but only 8 KV heads, which shrinks the KV cache 8×.
Unlock this and every premium problem across all tracks — plus hidden-test grading, advanced AI engineering modules and new content every week.
Start 7-day free trial