calculus

Frameworks do not differentiate formulas symbolically or estimate derivatives numerically. They differentiate the program. This lesson builds forward mode from dual numbers and reverse mode from the backward sweep, shows why a full gradient costs about four function evaluations at any size, and where the answer is not what you meant.

A deep network is a composition, so its derivative is a product of Jacobians. This lesson builds the chain rule from one variable up to matrix form, shows that the order you multiply that product in changes the cost tenfold, and explains vanishing gradients as an arithmetic consequence rather than a mystery.

One derivative becomes three objects once a function has many inputs and many outputs. This lesson builds the gradient, the Jacobian and the Hessian, shows what each one actually tells you, and explains why curvature decides how many steps an optimiser needs and why nobody ever writes the Hessian down.

Machine learning uses the derivative as a search strategy, not a symbolic exercise. This lesson builds it as the best local linear approximation, derives the gradient descent update from it, and shows why estimating derivatives numerically loses half your digits and costs one function evaluation per parameter.

Building the machinery: the linear space of allowed directions at a point, the inner product that gives it geometry, and why the Riemannian gradient is the ambient gradient projected rather than a new derivative.