kv-cache

A deep dive into how large language models actually run in production — why prefill is fast and decode is slow, how the KV cache works, sampling strategies like temperature and top-p, speculative decoding, and continuous batching with vLLM.

What the context window actually is, why 1M-token marketing doesn't mean what you think, the cost and accuracy curves that bend as you stuff more in, and the strategies (compaction, caching, retrieval) that keep long-context apps sane.