transformers
5 lessons tagged transformers: free, quiz-checked micro-lessons.
LLM Inference Internals: KV Cache, Sampling, and Serving at Scale
A deep dive into how large language models actually run in production — why prefill is fast and decode is slow, how the KV cache works, sampling strategies like temperature and top-p, speculative decoding, and continuous batching with vLLM.
Modern LLM Architectures: From Decoder-Only to Mixture-of-Experts
A deep tour of the architectural choices powering today's large language models — decoder-only Transformers, encoder-decoder designs, grouped-query attention, RoPE positional embeddings, and mixture-of-experts routing — with concrete numbers and trade-offs.
LLM Pretraining: Data, Loss, and What Actually Happens
A deep dive into how large language models learn from raw text: the next-token prediction objective, cross-entropy loss, the messy reality of web data curation (Common Crawl, dedup, quality filters), and the lineage from The Pile to FineWeb.
LLM Tokenization in Depth: BPE, Byte-Level BPE, and SentencePiece
A rigorous tour of how modern LLMs split text into tokens — covering byte-pair encoding, GPT-2/Llama's byte-level variant, SentencePiece, vocabulary design trade-offs, and why your tokenizer silently determines multilingual fairness, code quality, and arithmetic ability.
Attention and Transformers
From the limits of RNNs to the self-attention mechanism that replaced them. Learn how queries, keys, and values implement scaled dot-product attention, why multi-head attention captures richer structure, how positional encodings inject order, and how all of this assembles into a transformer block.