Llama, Mistral, Qwen and most modern LLMs encode token position with RoPE: instead of adding a position vector, they rotate each query and key vector by an angle that grows with position.
Unlock this and every premium problem across all tracks — plus hidden-test grading, advanced AI engineering modules and new content every week.
Start 7-day free trial