compression
4 free lessons tagged compression across Computer Science, Math. Each one is a short sequence of focused steps with narration and a five-question quiz at the end — take them in any order, no signup required.
Lossy Compression and the Rate-Distortion Tradeoff
Lossy compression is not lossless compression done badly. It is a deliberate trade of fidelity for rate, governed by a curve Shannon derived in 1959. This lesson covers rate-distortion theory, transform coding and the DCT, where JPEG actually discards information, perceptual coding in audio, motion compensation in video, and learned codecs.
Source Coding: Huffman, Arithmetic Coding and ANS
Entropy is a hard floor on lossless compression, and this lesson shows how coders approach it. Build a Huffman tree by hand, see exactly where it wastes bits on skewed sources, then follow the fix through arithmetic coding to asymmetric numeral systems, the entropy stage inside Zstandard.
Entropy: Measuring Information in Bits
Shannon's entropy measures the average surprise of a source, in bits, and it sets a hard floor on compression. Build it up from surprisal through joint and conditional entropy, mutual information, KL divergence and cross-entropy, with every number worked out by hand.
SVD and Least Squares
When there's no exact solution, project. When data is high-dimensional, compress. The SVD is the Swiss Army knife that does both — and more. Master orthogonal projection, the normal equations, the Singular Value Decomposition, low-rank approximation, and the pseudoinverse.