q-learning

Leave the model behind. Monte Carlo methods wait for a full episode to update; TD methods bootstrap from the very next step. See exactly where SARSA and Q-learning diverge on the on-policy/off-policy axis, and why that single difference changes everything about convergence guarantees.

Master the mathematical skeleton of reinforcement learning. Learn how the agent-environment loop formalizes decision-making, why the Markov property is the key assumption, and how Bellman expectation equations link policies to value functions.