Standard attention builds the full score matrix, which is huge for long contexts. FlashAttention never materialises it. Instead it processes keys in blocks, keeping a running max , a running normaliser and a running output for each query, and rescaling them as each new block arrives:
Unlock this and every premium problem across all tracks — plus hidden-test grading, advanced AI engineering modules and new content every week.
Start 7-day free trial