1 lesson tagged vocabulary: free, quiz-checked micro-lessons.
A rigorous tour of how modern LLMs split text into tokens — covering byte-pair encoding, GPT-2/Llama's byte-level variant, SentencePiece, vocabulary design trade-offs, and why your tokenizer silently determines multilingual fairness, code quality, and arithmetic ability.
