exploration
2 free lessons tagged exploration across AI. Each one is a short sequence of focused steps with narration and a five-question quiz at the end — take them in any order, no signup required.
Feedback Loops: The Model Trains on Clicks It Caused
A deployed recommender chooses its own future training data: it shows items, users respond to what was shown, and those responses become the next model's ground truth. This lesson maps the loop's consequences, exposure bias, popularity compounding, narrowing candidate pools, explains why offline metrics reward imitation of the loop, and covers the exploration budget that keeps the system learning.
Monte Carlo, TD, and Q-Learning
Leave the model behind. Monte Carlo methods wait for a full episode to update; TD methods bootstrap from the very next step. See exactly where SARSA and Q-learning diverge on the on-policy/off-policy axis, and why that single difference changes everything about convergence guarantees.