Lessons

A Gaussian process gives a mean and an uncertainty everywhere; the acquisition function turns them into one decision about where to spend the next expensive evaluation. This lesson covers Expected Improvement, GP-UCB and its no-regret guarantee, and Thompson sampling, and how each resolves exploration against exploitation.

Bayesian optimization needs a model that predicts the objective everywhere and knows how unsure it is. The Gaussian process delivers exactly that. This lesson builds the GP as a distribution over functions, shows how a kernel encodes assumptions, and explains why its posterior variance is the engine of the whole method.

Some functions cost hours or dollars to evaluate once, give no gradient, and must be optimized in as few tries as possible. This lesson sets up that problem, shows why grid and random search waste the budget, and introduces the surrogate-model loop that Bayesian optimization is built on.

A trie stores keys in their spelling rather than hashing them, which buys the one query a hash table cannot answer: find everything starting with this. This lesson covers the trie, the memory problem that makes it impractical, and the radix compression that fixes it and routes the internet.

A heap is the structure for when you need the smallest item repeatedly but never need the whole set sorted. This lesson builds the binary heap as an array, derives why building one costs linear rather than n log n time, and shows the top-k pattern that makes it worth knowing.

A binary search tree is elegant until sorted input turns it into a linked list. This lesson explains how balance is enforced: the rotation as the one legal repair, what red-black and AVL trees each guarantee, and why databases use B-trees with hundreds of children instead.

A hash table promises constant-time lookup, and the promise holds only because of how it handles collisions. This lesson builds one from the array up: hashing, chaining versus open addressing, why load factor is the tuning dial, and how modern tables scan sixteen slots at once.

Most real code never approaches a processor's arithmetic peak because it cannot be fed fast enough. The roofline model makes that concrete: plot operational intensity against achievable performance and the binding constraint becomes visible. This lesson covers the model, bandwidth versus latency, and the layout changes that follow.

DRAM is roughly two orders of magnitude further away than a register, so processors interpose several levels of cache. This lesson covers measured latencies at each level, cache lines and associativity, the three kinds of miss, and why identical algorithms differ tenfold based on access order alone.

Modern cores issue several instructions per cycle and execute them in whatever order their inputs become ready, while still appearing to run the program strictly in order. This lesson covers register renaming, the reorder buffer, the scheduler, and why the instruction window exists at all.

A processor does not execute one instruction at a time. It overlaps them in a pipeline, which raises throughput without making any single instruction faster. This lesson covers pipeline stages, the three classes of hazard, forwarding, the load-use stall, and why a mispredicted branch is expensive.

Lossy compression is not lossless compression done badly. It is a deliberate trade of fidelity for rate, governed by a curve Shannon derived in 1959. This lesson covers rate-distortion theory, transform coding and the DCT, where JPEG actually discards information, perceptual coding in audio, motion compensation in video, and learned codecs.

Shannon proved that a noisy channel still has a rate at which errors vanish. This lesson works a Hamming code by hand, follows Reed-Solomon into CDs, QR codes and deep space, reaches the capacity-approaching codes inside 5G, and ends on erasure coding versus replication in distributed storage.

Entropy is a hard floor on lossless compression, and this lesson shows how coders approach it. Build a Huffman tree by hand, see exactly where it wastes bits on skewed sources, then follow the fix through arithmetic coding to asymmetric numeral systems, the entropy stage inside Zstandard.

Shannon's entropy measures the average surprise of a source, in bits, and it sets a hard floor on compression. Build it up from surprisal through joint and conditional entropy, mutual information, KL divergence and cross-entropy, with every number worked out by hand.

Before a byte of HTTP moves, a name has to become an address. Follow the full resolution path from stub resolver to authoritative server, see how TTLs govern change, then trace head-of-line blocking from HTTP/1.1 through HTTP/2's TCP problem to HTTP/3 running on QUIC over UDP.

TCP turns a lossy packet service into an ordered byte stream, and almost every performance surprise on the internet comes from how it does that. Work through the handshake, cumulative ACKs, retransmission timers, the two windows, slow start and AIMD, CUBIC versus BBR, bufferbloat, and when to reach for UDP instead.

IP is the only layer the whole internet agrees on. Work through IPv4 and IPv6 addressing, CIDR subnet math by hand, what a routing table really holds, longest-prefix match, why NAT ended end-to-end addressing, and how one BGP announcement can pull a network off the internet.

A packet is a stack of nested envelopes, and the bottom layer has rules of its own. Work through encapsulation, Ethernet frames, MAC learning inside a switch, ARP, and MTU, then diagnose the classic failure where small requests succeed and large ones hang forever.

A file has no name; the name is a directory entry pointing at an inode. This lesson follows the kernel's I/O path: the VFS abstraction over every filesystem, the page cache and its writeback thresholds, buffered versus direct I/O, what fsync actually promises and what it does after a failure, ext4 journaling modes, and how a write finally reaches flash.

User code cannot touch a disk, a network card, or another process's memory without crossing into the kernel, and the crossing is not free. This lesson covers ring transitions and the syscall path, interrupts versus traps versus exceptions, measured boundary costs, why vDSO and io_uring exist, and how namespaces plus cgroups turn ordinary kernel features into containers.

Virtual memory is not a trick for pretending you have more RAM. It is the hardware and kernel machinery that gives every process a private, relocatable address space. This lesson covers multi-level page tables, why a TLB miss costs real cycles, demand paging, copy-on-write, mmap, swapping, huge pages, and how the OOM killer picks a victim.

A process is an address space plus a control block, and the thing Linux actually schedules is neither. This lesson walks the kernel side: what a context switch physically costs, how per-CPU run queues and preemption flags work, how nice values become weights, and why CFS was replaced by EEVDF in Linux 6.6.

Automations become infrastructure without anyone deciding they should. This lesson covers shadow automation and why banning it fails, documenting a flow so it survives its author, the signals that a workflow has outgrown no-code, and how to migrate without a rewrite.