gradient-descent
4 free lessons tagged gradient-descent across Math, AI. Each one is a short sequence of focused steps with narration and a five-question quiz at the end — take them in any order, no signup required.
Gradient descent: choosing the step and knowing the rate
Gradient descent is three lines of code and a hundred years of theory. This lesson derives why a safe step size is one over the smoothness constant, why the condition number governs everything, and why acceleration reaching order one over k squared is provably the best any first-order method can do.
The Derivative Is a Local Linear Model
Machine learning uses the derivative as a search strategy, not a symbolic exercise. This lesson builds it as the best local linear approximation, derives the gradient descent update from it, and shows why estimating derivatives numerically loses half your digits and costs one function evaluation per parameter.
Training: Optimization and Regularization
Go from a raw neural network to one that actually generalizes. Covers loss functions (MSE, cross-entropy), gradient descent variants (SGD, momentum, Adam), learning-rate effects, overfitting vs underfitting, and the regularization toolkit (L2/dropout/early stopping/batch norm).
Neural Networks and Backpropagation
Build intuition for how artificial neurons stack into layers, why nonlinear activations are non-negotiable, and how the chain rule turns a forward pass into exact gradients — illustrated with a tiny numpy forward+backward walk-through.