reward-modeling

2 lessons tagged reward-modeling: free, quiz-checked micro-lessons.

RLHF: From Christiano 2017 to InstructGPT to the Offline Era

Trace the full arc of Reinforcement Learning from Human Feedback — preference data, Bradley-Terry reward models, PPO with KL penalty, reward hacking, and why most labs have moved to offline alternatives like DPO and RAFT.

12 steps·~18 min

intermediate

LLM Post-Training: SFT, RLHF, DPO, and Modern Alignment Recipes

A deep dive into how raw pretrained language models become helpful assistants — from supervised fine-tuning on curated demonstrations, through reward modeling and PPO-based RLHF, to modern direct alignment methods like DPO and the recipes used in Llama 3, Llama 4, and DeepSeek.

12 steps·~18 min

RLHF: From Christiano 2017 to InstructGPT to the Offline Era

LLM Post-Training: SFT, RLHF, DPO, and Modern Alignment Recipes

Related topics