automatic-differentiation

Frameworks do not differentiate formulas symbolically or estimate derivatives numerically. They differentiate the program. This lesson builds forward mode from dual numbers and reverse mode from the backward sweep, shows why a full gradient costs about four function evaluations at any size, and where the answer is not what you meant.

Correct gradients are useless if computing them exhausts memory. Radiative backpropagation, path replay backpropagation's constant-memory trick, and why differentiable renderers needed their own compiler.

Differentiating a renderer is easy until geometry moves. Why silhouettes break naive automatic differentiation, and the three families of solutions: edge sampling, reparameterization, and warped-area methods.