shannon
2 free lessons tagged shannon across Computer Science. Each one is a short sequence of focused steps with narration and a five-question quiz at the end — take them in any order, no signup required.
Channel Capacity and Error-Correcting Codes
Shannon proved that a noisy channel still has a rate at which errors vanish. This lesson works a Hamming code by hand, follows Reed-Solomon into CDs, QR codes and deep space, reaches the capacity-approaching codes inside 5G, and ends on erasure coding versus replication in distributed storage.
Entropy: Measuring Information in Bits
Shannon's entropy measures the average surprise of a source, in bits, and it sets a hard floor on compression. Build it up from surprisal through joint and conditional entropy, mutual information, KL divergence and cross-entropy, with every number worked out by hand.