multimodal
5 free lessons tagged multimodal across AI. Each one is a short sequence of focused steps with narration and a five-question quiz at the end — take them in any order, no signup required.
End to End: What the Cascade Throws Away
Speech in, text, model, text, speech out is the standard architecture and it discards everything not in the words: emphasis, emotion, hesitation, overlap. End-to-end models keep it by never routing through text, and pay with a token rate roughly 185 times higher and far less training data. This lesson covers the trade, and why interleaving is the pragmatic answer.
Beyond Text: Multimodal, Safety, and Open Problems
Where discrete diffusion goes after language: unified multimodal models, sequence design in biology, the fixed-length problem nobody has cleanly solved, and why alignment and evaluation both need rebuilding.
Comparing LLM Capabilities: Reasoning, Code, Math, Multimodal
A capability-by-capability tour of frontier LLMs in 2026 — which models are strong at reasoning, code, math, long-context, multilingual, multimodal, and tool use, with hedged comparisons instead of point-estimate benchmark wars.
Multimodal AI: text, images, audio, video in one model
What "multimodal" actually means once you get past marketing copy. How modern models like GPT-4o, Gemini, and Claude blend modalities, and the design trade-offs (early vs late fusion, native vs adapted) you'll meet when building with them.
Vision-Language Models (VLMs): how machines read images
How models like CLIP, GPT-4V, and Claude visual learn to talk about pictures. Cover the contrastive trick behind CLIP, the difference between embedding models and generative VLMs, and where each one shines.