Quantizing LLMs: From 16 Bits to 4 Without Breaking the Model

A 70B model is 140 GB at 16 bits and fits on no single accelerator. At 4 bits it is 36 GB, runs about four times faster per token, and leaves most of the card free for KV cache. Getting there without wrecking the model is a specific, teachable skill. This path builds it from the arithmetic up: what quantization does to a number, why large transformers grow activation outliers that break naive methods, how GPTQ and AWQ use calibration data, and how to measure the damage honestly before you ship.