Optimization on Manifolds: When Parameters Live on a Curve

Rotations, subspaces, low-rank matrices and covariances are not vectors in flat space, and treating their constraints as penalties throws away structure that makes the problem easier. This path builds Riemannian optimization from the ground up: tangent spaces and the metric that decides what steepest means, the projection that turns an ordinary gradient into the right one, retractions that let you move along a curved space without solving differential equations, and why these non-convex problems are so often solved to global optimality anyway.