reinforcement-learning

A deep look at how reinforcement learning on chains-of-thought powers o1, DeepSeek-R1, Claude reasoning, and Gemini Thinking — covering GRPO, MCTS-style search, test-time compute scaling, and distillation into smaller models.

How Group Relative Policy Optimization turns unit tests and theorem provers into training signals — the technique behind DeepSeek-R1's leap in math and code reasoning without a reward model.

A deep dive into DPO (Rafailov et al. 2023) and its successors — how they reformulate RLHF as a classification problem, the math behind the implicit reward, and where each variant wins or loses against PPO-based pipelines.

Trace the full arc of Reinforcement Learning from Human Feedback — preference data, Bradley-Terry reward models, PPO with KL penalty, reward hacking, and why most labs have moved to offline alternatives like DPO and RAFT.

An honest map of RL in 2026 — the domains where it actually reaches production (LLM post-training, robotics policies, ad bidding, RLHF, reasoning models) and the places where it still cannot reliably cross the lab-to-deployment gap.

Tabular methods break down when the state space is continuous or astronomical in size. Learn how neural networks extend RL via DQN, how the policy gradient theorem makes it possible to differentiate through stochastic policies, and where actor-critic, PPO, and the deadly triad fit into the picture.

Leave the model behind. Monte Carlo methods wait for a full episode to update; TD methods bootstrap from the very next step. See exactly where SARSA and Q-learning diverge on the on-policy/off-policy axis, and why that single difference changes everything about convergence guarantees.

Master the mathematical skeleton of reinforcement learning. Learn how the agent-environment loop formalizes decision-making, why the Markov property is the key assumption, and how Bellman expectation equations link policies to value functions.

When you know the full MDP model, dynamic programming finds the optimal policy exactly. Learn the Bellman optimality equation, the contraction argument that guarantees convergence, and the concrete difference between policy iteration and value iteration — with a value-iteration code walkthrough.