kv-cache
4 free lessons tagged kv-cache across AI. Each one is a short sequence of focused steps with narration and a five-question quiz at the end — take them in any order, no signup required.
Measuring the Damage, and Shipping It
Quantization damage does not show up where people look for it. Perplexity barely moves while hard tasks degrade, and long reasoning suffers most because error compounds. This lesson covers building an evaluation that detects real loss, where the published cliffs are, KV cache quantization as a separate lever, end-to-end memory sizing, and the rollout that catches what evals miss.
Making Diffusion LLMs Actually Fast
Why bidirectional attention breaks the KV cache, how block-wise approximate caching brings it back, and the conditional-independence problem that decides how many tokens you can safely unmask at once.
LLM Inference Internals: KV Cache, Sampling, and Serving at Scale
A deep dive into how large language models actually run in production — why prefill is fast and decode is slow, how the KV cache works, sampling strategies like temperature and top-p, speculative decoding, and continuous batching with vLLM.
Context windows: tokens, limits, and "lost in the middle"
What the context window actually is, why 1M-token marketing doesn't mean what you think, the cost and accuracy curves that bend as you stuff more in, and the strategies (compaction, caching, retrieval) that keep long-context apps sane.