performance

A benchmark is an experiment, and most are badly designed enough to produce confident wrong answers. This lesson covers what a measurement must control to mean anything, the ways microbenchmarks lie including code the compiler deletes, how to catch regressions in continuous integration despite noisy machines, and how to recognise the point where optimising stops paying.

Slow software is slow for a short list of reasons, and each one has a signature you can recognise before you find the code. This lesson covers the six recurring bottleneck classes, waiting on I/O, chatty queries, allocation pressure, lock contention, memory access patterns and serialisation, with the symptom that identifies each and the fix that actually works.

A profiler is not a neutral observer: sampling and instrumentation see different things, distort the program in different ways, and answer different questions. This lesson covers how each works, why CPU time and wall-clock time give opposite answers, and how to read a flame graph correctly, including the axis that means nothing and is misread constantly.

Most optimisation effort is spent on code that was never the problem, and the reason is that intuition about where time goes is reliably wrong. This lesson covers why guessing fails, the arithmetic that caps what any optimisation can buy, the difference between latency and throughput, and how to set a target that tells you when to stop.

The pattern that made attention slow recurs across the stack, and once you know what to look for it is easy to find. This lesson applies the diagnosis to normalisation layers, optimizer steps, loss functions and inference decoding, covers why fused attention silently stops applying when a model deviates slightly from standard, and gives the profiling routine that decides where to look first.

The reason most teams never fuse anything is that CUDA asks you to manage threads, shared memory and synchronisation by hand. Triton moves the unit of programming from a thread to a block and hands the rest to a compiler. This lesson covers what that buys, what it still asks of you, how to decide a kernel is worth writing, and how to be sure it is correct.

For years attention was optimised by reducing FLOPs, and approximate methods that cut FLOPs kept failing to run faster. The reason is that attention was never compute-bound: it spends its time moving a matrix between GPU memory tiers. This lesson establishes that hierarchy, counts the traffic a standard implementation generates, and shows why an exact algorithm beat every approximation.

Two numbers govern where a pod lands and how it behaves under pressure, and they do completely different jobs. Requests are used for placement and are a promise; limits are enforced at run time and are a ceiling. This lesson separates them, covers the asymmetry between CPU and memory enforcement, and explains what actually happens when a node runs out.

Correct gradients are useless if computing them exhausts memory. Radiative backpropagation, path replay backpropagation's constant-memory trick, and why differentiable renderers needed their own compiler.

Why bidirectional attention breaks the KV cache, how block-wise approximate caching brings it back, and the conditional-independence problem that decides how many tokens you can safely unmask at once.

A hash table promises constant-time lookup, and the promise holds only because of how it handles collisions. This lesson builds one from the array up: hashing, chaining versus open addressing, why load factor is the tuning dial, and how modern tables scan sixteen slots at once.

Most real code never approaches a processor's arithmetic peak because it cannot be fed fast enough. The roofline model makes that concrete: plot operational intensity against achievable performance and the binding constraint becomes visible. This lesson covers the model, bandwidth versus latency, and the layout changes that follow.

DRAM is roughly two orders of magnitude further away than a register, so processors interpose several levels of cache. This lesson covers measured latencies at each level, cache lines and associativity, the three kinds of miss, and why identical algorithms differ tenfold based on access order alone.

Modern cores issue several instructions per cycle and execute them in whatever order their inputs become ready, while still appearing to run the program strictly in order. This lesson covers register renaming, the reorder buffer, the scheduler, and why the instruction window exists at all.

A processor does not execute one instruction at a time. It overlaps them in a pipeline, which raises throughput without making any single instruction faster. This lesson covers pipeline stages, the three classes of hazard, forwarding, the load-use stall, and why a mispredicted branch is expensive.

A model that runs is not a model that serves. This lesson covers what an inference server does that a naive loop cannot, continuous batching and why it dominates throughput, the memory arithmetic that decides which hardware you need, quantization for serving, and how to size a deployment from a traffic estimate.

Three ideas carry almost all production vector search. This lesson covers HNSW as a navigable small-world graph with its M and ef parameters, IVF as coarse partitioning with nprobe, and the quantization schemes that cut memory by an order of magnitude, plus how they compose into the hybrid indexes real systems actually run.

Why Unity runs physics on its own 50Hz clock instead of the frame rate, and what follows from that. Covers rigidbodies and colliders, moving things correctly with forces rather than the Transform, interpolation for smooth motion, tunneling and continuous collision detection, and where physics cost actually comes from.

A deep dive into how large language models actually run in production — why prefill is fast and decode is slow, how the KV cache works, sampling strategies like temperature and top-p, speculative decoding, and continuous batching with vLLM.

Understand where caches live, how cache-aside, read-through, write-through, and write-back differ, how to choose eviction policies, and how to prevent cache stampedes — with hit ratio math and a concrete cache-aside code snippet.

The five memory spaces a CUDA kernel can see and why they have wildly different speeds. Global vs shared vs constant vs local vs registers, coalesced access, bank conflicts, and a cheat-sheet table you'll actually reference.

Why GPUs eat CPUs for breakfast on some workloads and choke on others. Serial vs parallel execution, Amdahl's law, the latency-vs-throughput trade-off baked into silicon, and a rule of thumb for when to reach for a GPU.

Why your assets should never come from your origin. How a CDN's edge cache, geographic routing, and invalidation actually work, plus the cases where a CDN doesn't help (or quietly hurts).

The named caching patterns (cache-aside, read-through, write-through, write-behind), when each makes sense, and the failure modes that bite even experienced teams (thundering herd, stale invalidation, the second-hardest problem).

How to cut LLM bills 50-90% without sacrificing freshness. Covers the three caching layers (response, prefix, semantic), what each costs to build, and the cases where caching subtly breaks correctness.

Dive deep into the world of PostgreSQL indexes. Understand the core mechanics of B-tree, Hash, GIN, and GiST indexes, their optimal use cases, and how to choose the right indexing strategy to dramatically accelerate your database queries.

Explore Redis, a powerful open-source in-memory data store. Learn its core concepts, why it's used, its versatile data structures, and how it delivers blazing-fast performance for caching, real-time analytics, and more.