distillation
3 free lessons tagged distillation across AI. Each one is a short sequence of focused steps with narration and a five-question quiz at the end — take them in any order, no signup required.
When Generating Data Beats Collecting It
Synthetic data is roughly three orders of magnitude cheaper than human annotation, and cheapness is the least interesting thing about it. This lesson establishes what generation can and cannot manufacture: it produces coverage and format, never information the generator lacks, and the one exception is verifiable domains, where a checker turns generation into search.
Guidance and samplers: steering diffusion and making it fast
How raw denoisers become text-to-image systems: conditioning, classifier-free guidance and the CFG scale, latent diffusion, the sampler zoo from DDPM to DDIM and beyond, and the distillation techniques that cut a thousand steps down to a few.
RL in Reasoning Models: How o1, DeepSeek-R1, and Friends Think
A deep look at how reinforcement learning on chains-of-thought powers o1, DeepSeek-R1, Claude reasoning, and Gemini Thinking — covering GRPO, MCTS-style search, test-time compute scaling, and distillation into smaller models.