convergence

When you know the full MDP model, dynamic programming finds the optimal policy exactly. Learn the Bellman optimality equation, the contraction argument that guarantees convergence, and the concrete difference between policy iteration and value iteration — with a value-iteration code walkthrough.