mixture-of-experts

1 free lesson tagged mixture-of-experts across AI. Each one is a short sequence of focused steps with narration and a five-question quiz at the end — take them in any order, no signup required.

intermediate

Modern LLM Architectures: From Decoder-Only to Mixture-of-Experts

A deep tour of the architectural choices powering today's large language models — decoder-only Transformers, encoder-decoder designs, grouped-query attention, RoPE positional embeddings, and mixture-of-experts routing — with concrete numbers and trade-offs.

12 steps·~18 min

Modern LLM Architectures: From Decoder-Only to Mixture-of-Experts

Related topics