DPO aligns a language model with human preferences without training a separate reward model. For each prompt you have a chosen (preferred) and a rejected response, and the summed log-probabilities of each under the policy being trained and a frozen reference model.
Unlock this and every premium problem across all tracks — plus hidden-test grading, advanced AI engineering modules and new content every week.
Start 7-day free trial