probability

Combinatorics answers how many arrangements exist without producing any of them, which is what makes password strength, hash collisions and search-space size computable at all. This lesson builds the product rule, permutations, combinations, inclusion-exclusion and the pigeonhole principle, then applies them to problems where intuition is reliably wrong.

Shannon's entropy measures the average surprise of a source, in bits, and it sets a hard floor on compression. Build it up from surprisal through joint and conditional entropy, mutual information, KL divergence and cross-entropy, with every number worked out by hand.

Build the vocabulary that underlies all of ML: sample spaces, discrete and continuous random variables, PMFs, PDFs, and CDFs. Then tour the key distributions — Bernoulli, Binomial, Categorical, Gaussian, Poisson, Exponential, Uniform — with their parameters, mean, variance, and exactly when each appears in practice.

Master the three numbers that summarize any distribution: mean, variance, and standard deviation. Derive linearity of expectation, understand covariance and correlation, then see why the Central Limit Theorem makes the Gaussian unavoidable — with a worked numeric example from scratch.

Understand what it really means to update beliefs with data. Derive Bayes' theorem from first principles, dissect the roles of prior, likelihood, posterior, and evidence, work through a complete Beta-Binomial conjugate example numerically, and see why the base-rate fallacy trips up even experts.